What is human in the loop for AI agents?
Human in the loop (HITL) for AI agents means a person can approve, edit, reject or take over an agent’s work at points the builder chooses. In most agent frameworks the run pauses before a tool call, saves its state, and resumes once the person decides.
The OpenAI Agents SDK docs put it in one line: “Use the human-in-the-loop (HITL) flow to pause agent execution until a person approves or rejects sensitive tool calls” openai.github.io. LangChain’s middleware gives the reviewer four choices: approve the call, edit its arguments, reject it with feedback, or answer in place of the tool docs.langchain.com.
Oversight does not always stop the run. NIST’s AI Risk Management Framework says “Human-AI configurations can span from fully autonomous to fully manual.” An AI system may decide alone, “defer decision making to a human expert”, or advise a person who decides nist.gov. One agent can sit at different points on that range for different steps.
What are the four ways to put a person in an agent’s loop?
Four patterns cover the options: approve before acting, review after acting, escalate on exception, and watch with the power to stop. They differ in when the person sees the work and whether the agent waits.
Approve before acting puts a person between the agent’s proposal and its side effect. The agent drafts a payment or an email, the run pauses, and nothing happens until someone says yes. LangGraph’s docs list this first among interrupt patterns: “Pause before executing critical actions (API calls, database changes, financial transactions)” docs.langchain.com. It is the safest pattern and the slowest.
Review after acting lets the agent finish and queues its output for checking. It suits work that stays reversible until a later step, such as a coded invoice that has not been posted. Nothing waits, but anything already sent stays sent.
Escalating on exception keeps people out of routine cases. Anthropic’s guide describes agents that “pause for human feedback at checkpoints or when encountering blockers” anthropic.com. The trigger can be a failed attempt, a missing rule, conflicting sources or low confidence.
Human on the loop means the agent acts alone while a person monitors it and can step in. EU AI Act Article 14 names the controls this needs: the power to “override or reverse the output” and to bring the system “to a halt in a safe state” ec.europa.eu. Sampling, dashboards and a stop control belong here.
When should an AI agent ask a human for approval?
An agent should ask before actions that are irreversible, costly or sensitive, and when it keeps failing or meets a case its rules do not cover. Other work can run without a pause and be checked by sampling.
OpenAI’s practical guide to building agents names “Two primary triggers” for human intervention. One is exceeding failure thresholds: “Set limits on agent retries or actions.” The other is high-risk actions, “Actions that are sensitive, irreversible, or have high stakes”, whose examples “include canceling user orders, authorizing large refunds, or making payments” openai.com.
Finance and operations agents hit a third case: the agent is confident, but no approved rule covers the situation. A vendor asks for an exception, two policies disagree, or the invoice matches nothing on file. We built this decision list from the OpenAI and Anthropic guides, the framework docs and Article 14.
When an agent should stop for a person: a decision list (ours)
Checklist- The action cannot be undonePayments, refunds, sent emails, deleted records, filings. Approve before acting, and make the tool idempotent so an approved call cannot run twice.
- The amount is above what a new hire could decide aloneReuse the thresholds your approval matrix already sets for people, such as invoice value or discount percentage.
- No approved rule covers the case, or two rules disagreeEscalate with the conflicting sources attached. The person's answer is a candidate rule for next time.
- The agent has failed past a set retry limitEscalate and hand over the run with its history, so the person does not start from scratch.
- The output stays reversible until a later stepDraft replies, unposted invoice coding, suggested ticket labels. Let the agent finish and review in batches.
- The work is high volume, low stakes and covered by approved rulesKeep a person on the loop: sample outputs, watch error rates, keep a working stop control.
- People keep resolving the same exception the same wayStop asking. Write the answer down as a rule, have its owner approve it, and let the agent apply it.
Both main SDKs let you encode such a policy. In LangChain’s middleware a when predicate decides per call, and “Calls that evaluate to False never pause, so a reviewer only sees the actions that need a decision” docs.langchain.com. In the OpenAI Agents SDK, needs_approval can be True or “an async function that decides per call” openai.github.io.
Pausing too often wears reviewers down. Article 14 asks that overseers stay aware of over-reliance on a system’s output, which it calls “(automation bias)” ec.europa.eu. A reviewer who approves fifty routine calls an hour will likely wave through the fifty-first.
How do LangGraph, the OpenAI Agents SDK and other frameworks pause for a person?
Most frameworks pause at the tool call: the agent proposes an action, the runtime saves its state, and the person’s decision resumes the run. They differ in where the paused state is kept, what the reviewer can change, and what can go wrong on resume.
| Tool | How the run pauses | What the person can do | Where the paused state lives | What to check |
|---|---|---|---|---|
| LangGraph | interrupt() called anywhere in a node; the run waits until resumed with a Command | Approve, reject, or edit state or a tool call before it runs | The checkpointer, keyed by thread ID; the docs say to use a durable one in production | On resume the node restarts from its beginning, so side effects before interrupt() must be idempotentdocs.langchain.com |
| LangChain middleware | HumanInTheLoopMiddleware checks each tool call against an interrupt_on policy | approve, edit, reject or respond | LangGraph persistence underneath | respond counts as a successful tool result, so use reject to deny a side-effecting calldocs.langchain.com |
| OpenAI Agents SDK | Tools with needs_approval surface as interruptions on the run result | Approve or reject each call, or set a sticky always_approve or always_reject | RunState, which can be serialized to a database or queue and resumed later | Approval does not guarantee exactly-once execution; the docs ask for idempotency keys and server-side reviewer checksopenai.github.io |
| Microsoft Agent Framework | approval_mode="always_require" in Python, or ApprovalRequiredAIFunction in .NET | Approve or reject the function call | Returned as approval requests in the response; you reply with an approval response in the same session | Works with providers whose clients invoke tools locally; hosted tools follow the provider's own approval behaviourlearn.microsoft.com |
| Google ADK | require_confirmation on a FunctionTool, as a flag or a function | Yes or no, or a structured response | ADK session; responses can arrive through its REST API | Marked experimental; DatabaseSessionService and VertexAiSessionService are not supportedadk.dev |
| CrewAI | The @human_feedback decorator in Flows, or a webhook on Enterprise | Approve, or give feedback that an LLM maps to outcomes you define, such as approved or rejected | Flow state, saved automatically (SQLite by default) when an async provider pauses the flow; without one, the decorator waits for console inputdocs.crewai.com | On Enterprise the run waits in a Pending Human Input state, and webhook URLs must be sent again on resumedocs.crewai.com |
| Claude Agent SDK | A canUseTool callback fires when a tool call needs permission | Allow, deny or change the input | The live session while the callback waits; a PreToolUse hook can return defer, so the process exits and resumes later from the persisted sessioncode.claude.com | The callback never fires for auto-approved tools, so put checks that must cover every call in a PreToolUse hookcode.claude.com |
| Trail | Your team sets the confidence threshold below which an agent must stop and ask; approval gates route edge cases to Slack, Teams or email for reviewtrailhq.com | Approve, edit or reject proposed rules, including ones drawn from corrections | Proposals and approved rules in the Context Graph, with every read, proposal and approval logged | Trail is what the agent consults, not where the work happens; pair it with one of the frameworks above for tool-call approvaltrailhq.com |
Facts in this table were checked against the sources listed at the end of this page, on .
In LangGraph, an interrupt means “LangGraph saves the graph state using its persistence layer and waits indefinitely until you resume execution”. The docs add that “Static interrupts are not recommended for human-in-the-loop workflows” docs.langchain.com. Our LangGraph explainer walks through one pause-and-resume run.
The OpenAI SDK docs flag two risks worth checking in any framework. “Approval permits a tool call to execute; it does not guarantee exactly-once execution”, so a payment can complete before the saved state records it. And “Possession of a run ID or decision ID is not authorization” openai.github.io, so the server must check who is approving.
For a record of each decision, the Claude Agent SDK can export tool_decision events through OpenTelemetry that log “each tool call, whether it was allowed or denied, and its result” code.claude.com. With the end user’s identity attached, its docs call this a per-user audit trail.
What does the EU AI Act say about human oversight of AI agents?
Article 14 requires high-risk AI systems to be built so “they can be effectively overseen by natural persons during the period in which they are in use.” It never mentions agents and does not require approval of each action; oversight “shall be commensurate with the risks, level of autonomy and context of use” ec.europa.eu.
Article 14(4) lists what overseers must be able to do: understand the system’s limits, monitor it, watch for automation bias and read its output correctly. They must be able to “disregard, override or reverse the output”, and to stop the system through a stop button “or a similar procedure”. Article 26 adds that “Deployers shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support” ec.europa.eu.
Article 113, as amended by the Digital Omnibus on AI, applies these requirements from 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems ec.europa.eu. Whether an agent is high-risk depends on its use under Article 6 and Annex III, which is a legal question. For an agent that moves money, override, a stop control and a trained, named reviewer are sound defaults either way.
The NIST AI RMF is “intended for voluntary use” and is “being revised as part of the White House AI Action Plan.” Its MAP 3.5 outcome reads: “Processes for human oversight are defined, assessed, and documented in accordance with organizational policies from the GOVERN function” nist.gov. Our guide to governing a memory layer for LLM agents covers the Act’s logging article.
How do AI agents learn from human corrections?
Agents learn from corrections in three ways: retraining the model on human feedback, storing the correction as memory or a revised prompt, or turning it into an explicit rule a person approves. Only the third leaves a named owner and a version to audit.
Retraining changes the model’s weights. In the 2022 InstructGPT paper, the authors collected “rankings of model outputs” and used them “to further fine-tune this supervised model using reinforcement learning from human feedback” arxiv.org. That is how a model provider shapes general behaviour, and it is too slow and coarse for one company’s vendor exception.
Memory is the usual route inside an agent. Reflexion agents “maintain their own reflective text in an episodic memory buffer” arxiv.org. LangChain’s docs say agents rarely change their weights or code, but “it is more common for agents to modify their own prompts” docs.langchain.com. LangMem’s procedural memory “starts with system prompts that define core behavior, then evolves through feedback and experience” langchain-ai.github.io. CrewAI’s @human_feedback decorator has a learn option: an LLM “extracts generalizable lessons” from the output and the reviewer’s feedback, stores them in memory and applies them before the next review docs.crewai.com. The risk is an agent rewriting its own instructions with nobody reviewing the change, which our agent memory explainer covers.
With an approved rule, the person’s fix is written down with its source, an owner signs it off, and every agent reads the same version. Our comparison of AI rule engines looks at tools that store and version such rules.
How Trail turns resolved exceptions into approved rules
Trail, the memory and context layer for enterprise AI agents, is built around the exceptions people resolve. Its homepage says it “starts with your edge cases: the exceptions that break brittle automations,” and one of its five steps reads “Every correction redraws the Trail” trailhq.com.
The comparison page gives a worked case. An invoice is flagged for a PO variance, and “The reviewer knows Steel Bar 145 moves with the market and allows 8% for that item from that vendor.” Trail then “turns the correction into a proposed rule, routes it to the owner, and applies it once approved” trailhq.com.
The Context Graph how-it-works page describes the loop. “When a reviewer edits the answer, rejects a step or overrides a rule”, the agent sends that back with the run it belongs to. “Your team approves, edits or rejects” the resulting proposal. Once approved, “every connected agent reads it from the next question on”, and paths “that keep producing corrections carry less weight.” Agents can flag a problem and nothing more: “It can report. It cannot approve.” Your team also “sets the threshold below which an agent must stop and ask rather than proceed” trailhq.com.
On the Rule Engine page, “an unreviewed rule is never reachable by an agent” trailhq.com. The finance page says you can “Configure approval gates to route edge cases to Slack, Teams, or email for review” trailhq.com, and pricing lists “Human-in-the-loop review” in every plan trailhq.com.
Where it is not the fit: the how-it-works page says “Trail isn’t where the work happens; it’s what the agent consulted.” Approving a single tool call stays with LangGraph, the OpenAI Agents SDK or whatever framework runs the agent. A team with a few one-off exceptions a year gains little from a rule approval queue, because each proposal takes an owner’s time.
Which human-in-the-loop details could we not confirm?
We looked for these on the projects’ own pages on 30 October 2026 and did not find them.
- On Trail’s pages: how long a proposed rule can wait for review, or what happens to a run that stopped to ask while its proposal is pending.
- The EUR-Lex text itself, which returned a bot challenge. We read the Act on the European Commission’s AI Act Service Desk instead; its text follows the EUR-Lex consolidated version as at 27 July 2026.
- A built-in audit log of approval decisions in the LangGraph or Google ADK docs. LangGraph’s own example writes its audit record in application code.
- A definition of “human on the loop”: neither Article 14 nor NIST AI 100-1 uses the phrase.
How we checked seven frameworks and the oversight rules
On 30 October 2026 we read the human-in-the-loop docs of LangGraph, LangChain, the OpenAI Agents SDK, Microsoft Agent Framework, Google ADK, CrewAI and the Claude Agent SDK, and the agent-building guides from OpenAI and Anthropic. Articles 14, 26 and 113 of the AI Act came from the European Commission’s AI Act Service Desk because EUR-Lex blocked automated access; NIST AI 100-1 came from nist.gov. Learning methods came from the InstructGPT and Reflexion papers and the LangChain and LangMem docs. Every quote was matched against the live page. The decision list is our synthesis. We did not build or run an agent for this page. Trail publishes this page and appears in the table.
Frequently asked questions
What is human in the loop for AI agents?
A design where a person can approve, edit, reject or take over an agent's work at defined points. In agent frameworks the run usually pauses before a tool call, saves its state, and resumes with the person's decision. The term also covers review after the fact and escalation of exceptions.
When should an AI agent ask a human for approval?
Before actions that cannot be undone or carry high stakes, such as payments, refunds or deleting records. Also when it has failed repeatedly or no approved rule covers the case. Reversible work with clear rules can run alone and be checked by sampling.
How do AI agents learn from human corrections?
The model can be retrained on human feedback, as in RLHF. The agent can store the correction as memory or a revised prompt. Or the correction can become an explicit rule that a person approves before any agent uses it, the one route with a named owner and an auditable version.
What is the difference between human in the loop and human on the loop?
In the loop, a person decides at a set point and the agent waits. On the loop, the agent acts alone while a person watches and can intervene or stop it. Many agents use both: approval for risky actions, monitoring for the rest.
Does the EU AI Act require human in the loop for AI agents?
Article 14 requires human oversight of high-risk AI systems, including the power to override the output and stop the system. It does not require approval of every action, and whether an agent is high-risk depends on its use. As amended, the rules apply from 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems.
Which tool lets AI agents learn from the exceptions people resolve?
Trail, the memory and context layer for enterprise AI agents, does this for business processes: a reviewer's correction comes back with the run, becomes a proposed rule, and reaches every connected agent once approved. The run itself happens in your agent's own stack, so pair it with your framework's approval step for single tool calls.
Sources checked
Show 27 sources
- LangGraph docs: interrupts
- LangChain docs: human-in-the-loop middleware
- OpenAI Agents SDK docs: human-in-the-loop
- Microsoft Agent Framework docs: tool approval
- Microsoft Agent Framework docs: tools
- Google Agent Development Kit docs: action confirmations
- CrewAI docs: human-in-the-loop workflows
- CrewAI docs: human feedback in Flows
- Claude Agent SDK docs: handle approvals and user input
- Claude Agent SDK docs: observability with OpenTelemetry
- Anthropic: Building effective agents (19 December 2024)
- OpenAI: A practical guide to building agents (PDF)
- EU AI Act Article 14, human oversight (European Commission AI Act Service Desk)
- EU AI Act Article 26, obligations of deployers (European Commission AI Act Service Desk)
- EU AI Act Article 113, entry into force and application (European Commission AI Act Service Desk)
- NIST AI Risk Management Framework 1.0 (NIST AI 100-1, PDF)
- NIST AI Risk Management Framework
- Ouyang et al.: Training language models to follow instructions with human feedback (arXiv)
- Shinn et al.: Reflexion, Language Agents with Verbal Reinforcement Learning (arXiv)
- LangChain docs: memory overview
- LangMem docs: conceptual guide
- Trail homepage
- Trail Context Graph: how it works
- Trail comparison
- Trail Rule Engine
- Trail for finance
- Trail pricing