An AI agent receives a customer request, reads an account record and interprets the policy. It concludes that a refund is due and initiates payment. The conclusion is wrong because a recent exception was recorded in a document the agent could not access.
At that moment, “we had a human in the loop” is not enough. The business needs to reconstruct what happened and show who had authority.
The seven questions an audit trail must answer
- What did the agent receive? Record the relevant request, data and versioned context.
- What did it infer? Preserve the decision or recommendation and material reasoning trace without exposing unnecessary sensitive data.
- Which tools did it use? Identify searches, system calls and changes.
- Was the action within authority? Compare the action with the permission boundary in force at that time.
- Was approval required? Show whether a qualified person reviewed the evidence rather than simply confirming an automated result.
- Could the action be stopped? Record intervention, timeout and revocation controls.
- Who owned the outcome? Name the accountable role and the incident route.
Why a log is not automatically an audit trail
A large stream of technical events may be impossible for a business reviewer to interpret. Useful evidence connects the business instruction, data, permission, action, approval and outcome. It should be searchable, time-bound and protected against inappropriate alteration.
Meaningful human authority is a workflow property
A person who lacks time, evidence, competence or power to intervene is not exercising meaningful authority. Approval should be required at the point where judgement matters, and the interface should expose the exception rather than bury it.
The UK Government’s Data and AI Ethics Framework recommends named oversight, challenge routes and human intervention for risky or high-impact uses. The CMA’s AI-agent guidance makes the commercial consequence concrete: a business does not escape its consumer-law responsibilities because an agent took the action.
What should happen after the error?
Stop or constrain the workflow, preserve evidence, correct the customer impact, identify why the control failed, test the repair and decide whether authority can safely be restored. Incident learning should change the system—not merely close a ticket.
Direct answer: who is responsible when an AI agent is wrong?
Responsibility depends on the law, contract, sector and facts, but the deploying organisation cannot treat the agent as the accountable person. It must allocate human and organisational ownership, maintain evidence and provide intervention and redress.
Sources
Design identity, permissions, approval, evidence, intervention and revocation as one operational control system.
Use the meaningful human authority framework ↗