AI Trust Patterns

Human Oversight

Hard Gate

Ask people to confirm they've seen the AI's uncertainty before continuing, at high-stakes points only.

Phase
Design · Build
Applies to
End users, UX designers, developers
Oversight
Human in the loop
Key factor
Decision consequence
OverviewWhen to useQuestions to askExampleAnti-patterns to pay attention toReferencesEvidenceRelated

Overview

Some actions can't be easily undone: for example, a payment, a message to a customer, a clinical appointment, a decision about a person. A dismissible warning or superficial “approve or reject” turns oversight into a click, and an interface gate means nothing to the user.

Hard Gate makes explicit human confirmation a system constraint: the action can't happen until someone with the authority reviews and approves it. Forced Acknowledgment asks the person to actively confirm they understand the AI's stated uncertainty before moving on. It only works if it's rare, so keep it for high-stakes checkpoints.

When to use

Use when

  • High-stakes analysis or advice will be used for a decision.
  • The person isn't otherwise gated, or isn't the final approver.
  • The AI would make a decision about a person, money or similar.

Not needed when

  • The output is content people will read and work with. Blocking every summary causes fatigue; use a warning label.
  • Routine outputs. If people acknowledge it many times a day, it becomes a rubber stamp.

Questions to ask

Ask these at kickoff, when design work starts. Tick them off as your team answers.

Example

A content moderation agent flags a post as potentially violating platform policy. Its confidence score is 0.61, below the defined threshold of 0.75, so the post can't be actioned automatically. The reviewer sees the flagged content, the specific policy rule it may breach, and the reason the score fell below the threshold. Two actions are available, Remove or Dismiss, and neither is active until the reviewer opens the evidence panel. The decision, the reviewer's identity and the time on review are logged.

Content review · Flagged itemDo
Flag: Potential policy violationConfidence 0.61 / threshold 0.75
Policy rule§4.2 Harmful or misleading content
Reason below thresholdAmbiguous context: satire can't be ruled out
Evidence panel: the post, its thread and the account's history. Open evidence ▸
RemoveDismiss
Both actions unlock after you open the evidence. Reviewer: A. Kwame · Time on review is recorded with the decision.
See it in context · full screen
Content review console · Moderation queueIn context
Post #88199Confidence 0.61 / threshold 0.75
Policy rule§4.2 Harmful or misleading content
Why it's held 2Ambiguous context: satire can't be ruled out
Evidence: the post, its thread and the account's history Open evidence ▸
RemoveDismiss3
Reviewer: A. Kwame · Time on review recorded with the decision 4
  1. 1Above the threshold, items are actioned under the configured policy, with a notice and an appeal route. Only items below it are held for a person: here, 1 of 5. Keeping gates rare keeps them meaningful.
  2. 2The reason the AI is unsure is written in words, not just a score.
  3. 3Remove and Dismiss stay locked until the reviewer opens the evidence.
  4. 4The decision, the reviewer and the time on review are logged with the action.

Anti-patterns to pay attention to

  • Confirmation is optional or dismissible.
  • The gate can be skipped through a setting or the API.
  • The approver can't see what they're approving.
  • The wording is generic (“OK”, “Got it”) and doesn't name the uncertainty.
  • The acknowledgment appears on every output.

References

Current guidelines · EU AI Act, GDPR4 references

For orientation only, not legal advice. Some cases require explicit human confirmation. EU AI Act links open a readable version of the official text published on EUR-Lex; ISO/IEC 42001 is a paid standard.

  • EU AI Act Art. 14(4)(d) People overseeing high-risk systems must be able to decide not to use, or to disregard, override or reverse, the output.
  • EU AI Act Art. 14(4)(b) People overseeing the system should stay aware of automation bias. A deliberate acknowledgment interrupts it.
  • EU AI Act Art. 14(5) For remote biometric identification, no action is taken unless the result is separately verified by at least two qualified people: a legally mandated hard gate.
  • GDPR Art. 22 People have the right not to be subject to decisions made solely by automated systems with legal or similarly significant effects.

Evidence

Survey & researchSurvey insights

Survey insightIn the meeting-summary scenario, more than a quarter of respondents wanted the tool to block them until they had reviewed the flagged item, while most preferred to proceed with a clear flag. That's why Hard Gate is scoped to high-stakes actions, not every output: for content people read and work with, a visible flag is usually enough.

The survey is still open. Add your perspective ↗