AI Trust Patterns

Human Oversight

Monitoring Dashboard

A view for people monitoring an agent over time: what it did, what it flagged, what needs a person.

Phase
Operate
Applies to
Operators (human on the loop)
Oversight
Human on the loop
Key factor
System autonomy
OverviewWhen to useQuestions to askExampleAnti-patternAnti-patterns to pay attention toReferencesRelated

Overview

When an agent works on its own, someone still has to watch it. Being human on the loop means monitoring patterns and stepping in when something drifts, and a success-rate dashboard hides the cases that matter. The monitoring dashboard shows what the agent did, which mode it used, what it flagged, what people overrode, and how to stop it.

When to use

Use when

  • An agent acts on its own for some tasks, and someone is responsible for watching it.

Questions to ask

Ask these at kickoff, when design work starts. Tick them off as your team answers.

Example

The operations view for the intake agent.

Intake agent · TodayDo
128acted automatically
41sent for review
6open flags
3overridden
Reminder template mismatch · 4 patientsCheck: wrong clinic address
Missing-document requests · override rate 7% (usually 2%)Unusual
View overridesPause agent
See it in context · full screen
Operations console · Intake agentIn context
Intake agent · TodayPause agent4
128acted automatically
41sent for review 1
6open flags
3overridden
Open flags 2
Reminder template mismatch · 4 patientsCheck: wrong clinic address
Missing-document requests · override rate 7% (usually 2%) 3View reasons
  1. 1Automatic actions and items sent for review are counted separately.
  2. 2Open flags come first, in the same style they have everywhere else.
  3. 3Unusual patterns, like a jump in overrides, are called out with the reasons people gave.
  4. 4The agent can be paused from the same view.

Anti-pattern: Vanity metric

A single success rate. Everything looks fine until a patient calls.

Intake agent · TodayDon't
98%tasks completed

Anti-patterns to pay attention to

  • The dashboard shows totals but not open flags or overrides.
  • Automatic and reviewed actions aren't distinguished.
  • There's no way to pause the agent from the console.

References

Current guidelines · EU AI Act, ISO/IEC 420013 references

For orientation only, not legal advice. Applies to high-risk systems. EU AI Act links open a readable version of the official text published on EUR-Lex; ISO/IEC 42001 is a paid standard.