The Model Was Right. Nobody Trusted It.

01The answer nobody could check

It is the pattern we meet more often than any other. A policy assistant in a regulated organisation answers correctly most of the time — measurably, repeatably. It is also barely used, because every answer arrives as a confident paragraph with nothing behind it, and the people asking are the ones who will be held accountable if it is wrong. So they look the answer up anyway. The assistant has added a step to their day instead of removing one.

The remedy is rarely a better model. It is a design decision: every sentence linked to the clause it came from, the source opened beside the answer, and the date that policy last changed shown before anyone has to ask. Checking goes from a search to a glance. People stop checking every answer — not because they were told to trust the system, but because they can finally see when to.

02Confidence is a design material

Models are never certain, and interfaces almost always pretend they are. The same typeface, the same colour and the same calm tone carry an answer the system is sure of and one it has essentially guessed. People learn, correctly, that they cannot tell the difference — and so they treat every answer as a guess.

Designing uncertainty honestly changes that. A plain 'I could not find this in the approved sources' instead of an invention. Suggestions phrased as suggestions. Doubtful cases routed to a person with the reason attached. It can feel like admitting weakness. In practice it is the most reliable way we know to build trust, because a system that says 'I don't know' on the hard question is believed on the easy ones.

A system that admits it does not know is trusted on the days it does.

03The chat box was the wrong shape

When generative AI arrived, a great many enterprise tools grew a chat window almost overnight. It was the quickest thing to build and, often, the wrong thing to use. An inspector on a site walk does not want to compose a prompt wearing gloves. An analyst closing the month does not want a conversation; they want the three exceptions that need a decision, already explained.

Human-centred research is what reveals the right shape. Sometimes it is a conversation. More often it is a form the AI has already filled for a person to confirm, a suggestion inline in the screen they already use, a photograph taken and a finding drafted, or an agent working quietly in the background and surfacing only what needs a human. The intelligence underneath is identical. The experience decides whether it saves an afternoon or wastes one.

• 01Ask what the person is trying to finish, not what they might type.
• 02Put the AI where the work already happens, not in another tab.
• 03Use conversation where the question is open, and structure where it is not.
• 04Measure whether the task got easier, not whether the feature got clicks.

04Why we design before we build

Every one of these lessons is cheap to learn on a prototype and expensive to learn in production. That is why our AI programmes put researchers and designers beside engineers from the first week — watching the work, sketching the interaction and testing a realistic prototype with the people who will use it, before the architecture is fixed.

It is not a finishing coat. The moment an experience becomes simple is usually the moment a data model, an API or a permission has to change, and that conversation is only affordable early. Designing UX-first is not slower. It is the difference between an AI system that is approved and one that is adopted — and only the second ever pays for itself.