Trust and Transparency

Data Freshness Signal

Show reviewers when the information the agent used may be out of date, before they act on it.

Phase: Design · Build
Applies to: End users, developers
Oversight: Human in the loop
Key factor: Decision consequence

Overview When to use Questions to ask Example Anti-pattern Anti-patterns to pay attention to References Related

Overview

An agent can produce a perfectly confident answer based on data that's no longer correct. Policies change, records update, and reference data has a shelf life. Confidence scores measure model certainty; they don't measure whether the underlying data is current. A user who sees a high-confidence result has no way to know the information was last updated six days ago, unless the interface tells them. Data Freshness Signal shows the age of the data at the point of review, not buried in system logs.

When to use

Use when

The agent acts on data that changes over time: policies, eligibility rules, records, reference data.
A reviewer is about to approve a result where outdated data would change the decision.
There's a known sync interval or last-update timestamp the system can surface.

Not needed when

The data source is static and doesn't change between sessions.
The output is low-stakes and easily corrected after the fact.

Questions to ask

Ask these at kickoff, when design work starts. Tick them off as your team answers.

Which data sources does the agent draw on, and how often does each update?
At what point does data become meaningfully stale? An hour matters for some decisions; a week may not matter for others.
Where does the reviewer make the decision that depends on this data? The freshness signal belongs there, not in a settings panel.
What does the reviewer do when data is stale: trigger a refresh, or escalate?
Is there a hard threshold beyond which the action should be blocked until data is refreshed?

Example

A reviewer approves an AI-prepared eligibility assessment. Next to the result, a label shows how old the policy rules are, so the reviewer can check before approving.

Eligibility assessmentDo

ResultLikely eligible

Policy rulesLast updated 6 days ago Check for revisions

Check for revisionsApproveReturn

If the data's age crosses a defined threshold, the label escalates.

Eligibility assessment · Stale dataDo

ResultLikely eligible

⚠ Policy data hasn't been refreshed in 14 days. This result may not reflect current rules.

Verify before approvingApproveReturn

Anti-pattern: Confidence without freshness

The agent produces a high-confidence result, and the reviewer approves. What neither the score nor the interface showed: the policy rules were last updated 14 days ago, and a revision went live 4 days ago. The agent was certain about rules that were no longer current, and the reviewer had no signal to act on.

Eligibility assessmentDon't

ResultLikely eligible Confidence: 0.88

ApproveReturn

Anti-patterns to pay attention to

Confidence is shown, but no data age or last-update time is visible at review time.
The last-update timestamp exists in system logs but isn't shown in the review interface.
Stale data looks exactly like fresh data: no badge, no label, no distinction.

References

Current guidelines · EU AI Act, ISO/IEC 420013 references

For orientation only, not legal advice. EU AI Act links open a readable version of the official text published on EUR-Lex; ISO/IEC 42001 is a paid standard.

EU AI Act Art. 13(3)(b) Instructions for use must cover the system's characteristics, capabilities and limitations. Data staleness is a known limitation to communicate where it affects decisions.
EU AI Act Art. 14(4)(c) People overseeing the system must be able to interpret output correctly. A result built on stale data can't be interpreted correctly without knowing the data's age.
ISO/IEC 42001 A.7.4 Set and verify explicit data-quality criteria for production data. Being current enough for the decision is one of them.