)
Last updated: October 2026
What does human in the loop mean for AI agents, and when should a person approve what an agent does?
Human in the loop (HITL) means a person can review, approve or reverse an agent's action before it takes effect. It works when approvals are rare, tiered by risk and backed by hard technical limits. Blanket approval fails: according to Anthropic (2026), users approve 93% of Claude Code permission prompts.
TL;DR: Key Takeaways
- Users approve 93% of Claude Code permission prompts, a pattern Anthropic describes as approval fatigue (Anthropic, March 2026).
- Auto-approve use grows from about 20% of sessions for new users to over 40% at 750 sessions, so oversight shifts from per-action review to monitoring (Anthropic, February 2026).
- OWASP names excessive autonomy, meaning no independent approval of high-impact actions, as one of three root causes of Excessive Agency (OWASP LLM06:2025).
- The Model Context Protocol specification says there should always be a human in the loop with the ability to deny tool invocations (MCP specification).
- EU AI Act obligations for Annex III high-risk systems, including Article 14 human oversight, now apply from December 2, 2027 under Regulation (EU) 2026/1744 (Cloud Security Alliance, 2026).
- Gartner forecasts that over 40% of agentic AI projects will be cancelled by the end of 2027, partly on insufficient risk controls (Gartner via IT Brief, 2025).
Oversight models at a glance
| Oversight model | Human role | Best for | Main weakness |
|---|---|---|---|
| Human in the loop | Approves each high-impact action before it runs | Irreversible, financial or regulated actions | Approval fatigue and latency |
| Human on the loop | Watches a live stream and can intervene | High-volume, reversible actions | Alerts get missed |
| Human in command | Sets policy, budgets and limits in advance | Whole fleets of agents | Policy gaps surface late |
| Human out of the loop | Reviews outcomes after the fact | Low-risk, sandboxed tasks | No chance to stop damage |
What is human in the loop for AI agents?
Human in the loop for AI agents is a control pattern where a person must approve, edit or reject an agent's proposed action before the action executes. It differs from reviewing outputs afterward. The approval sits inside the execution path, so a "no" stops the action.
The phrase covers three distinct patterns, and teams often blur them. In the loop means the agent pauses and waits. On the loop means the agent keeps running while a person watches and can step in. Out of the loop means people review results later, through sampling or audits.
Agents make the distinction urgent because they act. A chatbot that drafts a bad answer wastes a minute. An agent with a payments tool, a mailbox and a production database can move money, email customers and delete records before anyone reads a log.
How we researched this guide
We built this guide from primary sources: Anthropic's research, OWASP guidance, the MCP specification, EU AI Act summaries and Gartner's forecast. Vendor-reported figures are labeled, and secondary sources are flagged.
Why approving every action fails
Approving every action fails because people stop reading prompts they see all day. Anthropic reports that users approve 93% of Claude Code permission prompts, and it calls the result approval fatigue. A control that is approved nearly every time no longer separates safe actions from risky ones.
Anthropic's agent autonomy research (2026) found that new users, with fewer than 50 sessions, use full auto-approve in roughly 20% of sessions, while users with 750 sessions pass 40%. Experienced users also interrupt more often, about 9% of turns versus about 5%, because they have moved from approving each step to watching and stepping in.
Anthropic's answer to fatigue is a classifier. Its auto mode lets a model screen each action. Anthropic's engineering team (2026) reports a 0.4% false positive rate on real traffic but a 17% false negative rate on real overeager actions. Roughly one in six risky actions passes. That is a good trade for a coding assistant in a sandbox. It is a poor trade for an agent that can issue refunds or change IAM roles.
Gartner sees the same tension at the program level. Gartner (2025) expects over 40% of agentic AI projects to be cancelled by the end of 2027, in part because of insufficient risk controls.
| Failure mode | What happens | Result |
|---|---|---|
| Approval fatigue | Reviewers click through prompts | Risky actions pass unnoticed |
| Alert fatigue | Monitors ignore a noisy stream | Late intervention |
| Blanket blocking | Every action waits for a person | Agents deliver no value |
| Classifier-only gating | A model approves its own kind | A 17% miss rate on overeager actions in Anthropic's test |
)
Which agent actions should require human approval?
Require approval for actions that are irreversible, move money, touch regulated data, change permissions or cross your trust boundary. Let read-only and easily reversible actions run without a prompt. OWASP's guidance recommends human approval for high-impact actions and least privilege for everything else.
OWASP (2025) defines Excessive Agency through three root causes: excessive functionality, excessive permissions and excessive autonomy. Approval addresses only the third. If an agent holds tools it does not need, or credentials broader than its task, a human prompt is a weak backstop. Remove the capability first, then gate what remains. Our guide to excessive agency covers the first two causes in depth.
Anthropic's incident log offers a way to sort actions. Its classifier targets four families: destroying or exfiltrating data, degrading security posture, crossing trust boundaries, and bypassing review. Each family maps to a tier below.
| Tier | Example agent actions | Control |
|---|---|---|
| 0: read-only, internal | Search a knowledge base, summarize a ticket | No approval, log only |
| 1: reversible writes | Draft an email, create a ticket | Run, notify, allow undo |
| 2: external or financial effects | Email customers, issue a refund, book travel | Approve before it runs |
| 3: destructive or privilege changes | Delete records, change IAM roles, deploy to production | Named approver plus a second reviewer |
| 4: prohibited | Disable logging, edit the agent's own permissions | Block at the gateway, no approval path |
Tier 4 actions should never reach a human prompt. A persuasive request to disable logging is exactly what an attacker or a confused agent would generate, so policy blocks it outright.
How to design oversight that scales
Oversight scales when approvals are rare, informative and backed by limits the model cannot talk its way around. Design for the few decisions that need judgment, automate the rest through policy, and measure whether reviewers are still paying attention.
Tier approvals by risk
Start with the table above. Assign each tool, and each tool parameter that changes impact, to a tier. A "send email" tool might be tier 1 for internal recipients and tier 2 for external ones. Tiering by parameter keeps the approval count low without weakening the controls that matter.
Show the approver what will actually happen
Show the exact tool, parameters and data affected, not a generic "agent wants to run a tool" message. n8n's documentation describes this pattern for AI agent tool calls: the workflow pauses, an approval request goes to a channel such as Slack, Teams or Gmail with the tool and parameters the AI chose, and a denial cancels the action and tells the agent.
Make the default safe
If nobody answers, deny. Add timeouts, spend limits and rate limits so that a stuck or looping agent cannot exhaust the approver or the budget. Our post on rate limiting AI agents shows how to set these at the gateway.
Enforce limits outside the model
Deterministic controls such as allowlists, scoped credentials and network egress rules do not depend on a person or a model behaving well. Resilient Cyber argues that per-action approval cannot be the primary safety control and that boundaries should be enforced by infrastructure. We agree with the ordering: limits first, humans for exceptions. See RBAC for AI agents for scoping tool access per identity.
Measure the approvers
Track approval rate, time to decision and override rate per approver and per tool. A near-100% approval rate with decisions in under two seconds signals a rubber stamp. The EU AI Act summary at euaiact.com notes that overrides should be logged under Article 12, and that override rates can be calculated from those logs.
| Design pattern | What it solves | Failure mode to watch |
|---|---|---|
| Risk tiers per tool and parameter | Cuts approval volume | Tiers drift as tools change |
| Exact-action approval prompts | Makes review informative | Prompt text is attacker-influenced |
| Gateway-enforced allowlists | Removes decisions from the loop | Policy gaps for new tools |
| Approver metrics and sampling | Detects rubber-stamping | Metrics ignored by owners |
What the EU AI Act asks of human oversight
The EU AI Act requires high-risk AI systems to be designed so people can oversee them while in use. Article 14 sets five capabilities for the assigned overseer, and Article 26(2) tells deployers to give that person competence, training, authority and support. Most enterprise agents are not automatically high-risk.
euaiact.com summarizes Article 14(4). The measures must let the assigned person understand the system's capacities and limits, stay aware of automation bias, interpret the output, decide to disregard or reverse it, and stop the system through "a stop button or a similar procedure." The page stresses that this is a design duty, not only a staffing policy.
On timing, the Cloud Security Alliance (2026) reports that Regulation (EU) 2026/1744, the Digital Omnibus on AI, entered into force on July 27, 2026. It moved Annex III high-risk obligations from August 2, 2026 to December 2, 2027, and Annex I product-embedded systems to August 2, 2028. Article 5 prohibitions and general-purpose AI obligations were not changed. This is a summary, not legal advice: confirm scope against the consolidated regulation text with counsel.
| Article 14(4) capability | What it means for agent design |
|---|---|
| Understand capacities and limits, and monitor | Document what the agent can do, and expose anomaly signals |
| Stay aware of automation bias | Train approvers, and track approval and override rates |
| Interpret the output | Show the agent's proposed action and reasoning in plain terms |
| Disregard, override or reverse | Let the overseer decide without escalating first |
| Stop or interrupt | Provide a kill switch that halts the agent in a safe state |
Security and governance: where human oversight breaks
Human oversight breaks when approvers cannot trust what they see or cannot keep up with volume. Prompt injection can shape the request in front of them, sensitive data can leak through approval payloads, and tool misuse can be split into many small steps that each look harmless.
Prompt injection shapes what the approver sees
An agent that reads web pages, email or documents can ingest hidden instructions. If the approval prompt is generated from the agent's own summary, an attacker influences the very text the reviewer relies on. Our indirect prompt injection guide shows how this works in practice. Approvals therefore need to show raw tool parameters, not only the agent's description of them.
Data leaks and small-step misuse
Approval requests travel through chat tools and inboxes, so customer records or secrets in the prompt turn the review channel into a leak path. Ten approved calls that each export a few rows can add up to a full export, and per-action review does not see the pattern. Redact sensitive fields, and use rate limits, volume caps and sequence-aware detection.
How NeuralTrust addresses this
NeuralTrust treats oversight as a runtime control problem. Agent Gateway (TrustGate) sits between agents, models and MCP tools. It enforces per-agent tool permissions and limits before a call executes, so tier 4 actions are blocked and tier 3 actions can be routed for approval. Agent Runtime Security (TrustGuard) inspects traffic for prompt injection and data leakage as it happens. Together they form the Runtime Security Mesh: policy decides what is allowed, detection catches what policy cannot predict, and people see only the exceptions worth their attention. Our control plane overview explains the architecture, and MCP Security 101 covers the tool layer.
A human-in-the-loop checklist for AI agents
Use this checklist before an agent reaches production. It turns the sections above into decisions you can assign and audit.
- Inventory every tool the agent can call, and remove any it does not need.
- Assign each tool and each impact-changing parameter to a risk tier.
- Block tier 4 actions in policy so no approval path exists.
- Require approval for tier 2 and tier 3, with a named approver for tier 3.
- Show the exact tool call and parameters in every approval prompt.
- Deny by default on timeout, and cap spend, volume and rate.
- Log every approval, denial and override with the approver identity.
- Review approval rate, decision time and override rate monthly.
- Red-team the gate before launch and after each tool change.
Which oversight model should you choose?
Choose the model by action risk, not by agent type. Use human in the loop for irreversible and regulated actions, human on the loop for high-volume reversible work, and human in command for fleets. Most production agents mix all three.
| Your situation | Recommended model | Why |
|---|---|---|
| Hiring, credit or health decisions that may be high-risk under the AI Act | Human in the loop, with logging | Article 14 expects override and stop capability |
| Customer-facing agent that issues refunds | Approval above a threshold, monitoring below it | Keeps volume low while capping loss |
| Internal coding agent in a sandbox | Human on the loop with sandbox limits | Fast iteration, bounded blast radius |
| Fleet of agents across teams | Human in command with gateway policy | One policy point beats many prompts |
Conclusion
Human in the loop is a control to spend carefully. Anthropic's 93% approval rate shows what happens when it is spent on everything. Tier your agent's actions, enforce hard limits outside the model, show approvers the exact call, and measure whether review is still real. Reserve people for the few decisions where judgment matters, and let policy handle the rest.
Secure AI Agents in Production with NeuralTrust
Put policy, detection and audit between your agents and every tool they can reach, so human review is reserved for the decisions that need it.
)
Related Reading
- Excessive Agency in AI agents
- RBAC for AI agents
- OWASP Top 10 for Agentic Applications 2026
- MCP Security 101
FAQs about Human in the Loop AI Agents
1. What is human in the loop in AI agents?
Human in the loop means a person must approve, edit or reject an agent's proposed action before it runs. The approval sits in the execution path, so a denial stops the action. Reviewing logs afterward is oversight after the fact, not a control.
2. What is the difference between human in the loop and human on the loop?
In the loop, the agent pauses and waits for approval. On the loop, the agent keeps working while a person monitors a live stream and can intervene. The first suits irreversible actions. The second suits high-volume, reversible work where pausing every step would erase the agent's value.
3. When should an AI agent ask for human approval?
Ask when an action is irreversible, moves money, touches regulated data, changes permissions or crosses a trust boundary, such as issuing refunds, deleting records or deploying to production. Read-only and reversible actions can run without a prompt, which keeps approval volume low enough for reviewers to stay attentive.
4. Does the EU AI Act require human in the loop?
Article 14 requires high-risk AI systems to be designed for effective human oversight, including the ability to override and stop the system. It does not use the phrase "human in the loop." Under Regulation (EU) 2026/1744, Annex III obligations apply from December 2, 2027. Confirm whether your agent is high-risk with counsel.
5. What is approval fatigue in AI agent oversight?
Approval fatigue is approving prompts without reading them because there are too many. Anthropic reports that users approve 93% of Claude Code permission prompts and describes people no longer paying close attention. Risk tiering, fewer prompts and deny-by-default timeouts reduce it.
6. Can an AI classifier replace human approval?
For low-risk work, a classifier can reduce prompts. For high-impact actions, it is not a safe substitute. Anthropic reports a 0.4% false positive rate for its auto mode on real traffic but a 17% false negative rate on real overeager actions. Keep human approval or hard policy blocks for tiers 3 and 4.
7. How do you implement human in the loop for MCP tool calls?
The MCP specification says there should always be a human in the loop with the ability to deny tool invocations, and that clients should show tool inputs before calling a server. In practice, route calls through a gateway that enforces tool permissions, requests approval for high-tier tools and logs each decision.
About the Author
Roger Howroyd is Head of Global SEO and AI at NeuralTrust, where he leads the company's search strategy across SEO, AEO, GEO, and LLM optimization. He specializes in AI-powered search, content strategy, and SEM. Connect on LinkedIn.
NeuralTrust is the leading platform for securing and scaling AI agents. Named a Pioneer in the Gartner Emerging Market Quadrant for AI Application Security 2026, recognized across four Gartner Hype Cycle reports in 2026, and featured in the Gartner Market Guide for Guardian Agents 2026, the Gartner Market Guide for AI Gateways 2025 and the KuppingerCole Leadership Compass for Generative AI Defense 2025. Headquartered in Barcelona with offices in London and New York. ISO 27001 certified.
Sources
- Anthropic Engineering, "Claude Code auto mode," Mar 25, 2026: https://anthropic.com/engineering/claude-code-auto-mode
- Anthropic Research, "Measuring AI agent autonomy in practice," February 18, 2026: https://www.anthropic.com/research/measuring-agent-autonomy
- OWASP GenAI, "LLM06:2025 Excessive Agency": https://genai.owasp.org/llmrisk/llm06/
- Model Context Protocol, "Tools" specification (2026-07-28 version): https://modelcontextprotocol.io/specification/latest/server/tools
- euaiact.com, "EU AI Act Human Oversight": https://www.euaiact.com/blog/eu-ai-act-human-oversight
- Cloud Security Alliance, "EU AI Act High-Risk Deadline": https://labs.cloudsecurityalliance.org/research/csa-research-note-eu-ai-act-high-risk-deadline-omnibus-20260/
- IT Brief on Gartner's agentic AI forecast, Jun 24, 2025: https://itbrief.news/story/over-40-of-agentic-ai-projects-set-for-cancellation-by-2027
- n8n Docs, "Human-in-the-loop for AI tool calls": https://docs.n8n.io/build/integrate-ai/ai-examples/human-in-the-loop-for-tools
- Resilient Cyber, "The Human-in-the-Loop Illusion": https://resilientcyber.beehiiv.com/p/the-human-in-the-loop-illusion
)
)
)
)