Trust and Transparency

Confidence Translation

Express how certain the agent is in words a reviewer can act on, not as a raw number.

Phase: Design · Build
Applies to: End users, content designers
Oversight: Varies with stakes
Key factor: Decision consequence

Overview When to use Questions to ask Example Anti-pattern Anti-patterns to pay attention to References Evidence Related

Overview

A raw confidence score (0.82, 64%, p=0.71) tells a reviewer very little. People respond to words, not numbers: “Likely yes, but check this first” prompts a different action than “71%”. When confidence is expressed only as a number, high scores suppress scrutiny and low scores cause confusion without direction. Confidence Translation turns the signal into language that tells the reviewer what to do, not just what was calculated.

When to use

Use when

The agent produces output a person will act on, and confidence varies across outputs.
Stakes are high enough that a wrong answer has consequences.
End users include people who aren't expected to interpret probabilities.

Not needed when

Every output is confirmed by an external source before action, so the signal adds nothing.
The output is a brainstorm or suggestion. Ideas aren't facts; no signal is needed.

Questions to ask

Ask these at kickoff, when design work starts. Tick them off as your team answers.

How would an experienced end user describe this output in conversation: “almost certain”, “likely but check”, “unclear”? Use that language.
Which outputs get no signal, a quiet label, or a full plain-language reason?
What does a low-confidence result tell the end user to do, not just that something is uncertain?
Where does technical detail live if an engineer or auditor needs it? One level down, not in the primary view.
What would turn a low-stakes task into a high-stakes one, for example if the output is forwarded, published or acted on by someone downstream?

Example

The same assistant handles two requests from the same user, with different stakes.

Assistant · Two requestsDo

“What are some team names for the offsite workshop?”

Wavelength · Common Ground · Open Loop · Basecamp

No confidence signal. These are suggestions, not facts.

“Is this contractor eligible to work on this project, given their current clearance level?”

Likely yes, based on the current clearance record, but the clearance was last reviewed 11 months ago.

Confirm the renewal status before assigning. View clearance record

The second response tells the reviewer what it found, why it's uncertain and what to check. It doesn't make them interpret a percentage.

Anti-pattern: Raw mechanics

A badge with the model's internal output. Technically accurate, and useless to a reviewer. Half will ignore it; the other half will treat the number as a verdict without understanding what it measures. Neither is oversight.

AssistantDon't

“Is this contractor eligible to work on this project?”

p(clearance_valid)=0.62 < τ=0.70 · last_review_delta=335d

Anti-patterns to pay attention to

Probabilities or thresholds shown with no plain-language framing.
The signal tells the reviewer a score but not what to do with it.
Technical detail is in the primary view, not one level down.

References

Current guidelines · EU AI Act, ISO/IEC 420015 references

For orientation only, not legal advice. EU AI Act links open a readable version of the official text published on EUR-Lex; ISO/IEC 42001 is a paid standard.

EU AI Act Art. 14(4)(c) People overseeing high-risk systems must be able to interpret the output correctly. Plain language is what makes correct interpretation possible for non-technical users.
EU AI Act Art. 14(3) Oversight measures must be commensurate with risk, level of autonomy and context of use. The tiers (no signal, quiet label, full plain-language reason) are the design expression of this proportionality.
EU AI Act Art. 9 Risk management identifies and evaluates risks. The risk levels from that assessment decide which tier of confidence signal applies.
EU AI Act Art. 4 Providers and deployers must ensure sufficient AI literacy among the staff who operate and use the system. Plain-language signals support that in practice.
ISO/IEC 42001 A.8.2 Give users plain-language information about the system's capabilities and limits.

Evidence

Survey & researchSurvey insights, respondent quotes

“When working with AI, I find that a descriptive output is more reliable than a numeric confidence threshold. As that’s a better reflection of how LLMs actually reason - in language.”
Survey respondent, open-ended answer

Survey insightMost respondents would rather see an explanation or the sources than a percentage. Their reaction to the mechanics depended on how they were explained: the largest group said knowing how confidence is calculated would only help if it were in plain language.

The survey is still open. Add your perspective ↗