Human in the loop: where a person approves the AI’s work
Human in the loop means a person reviews and approves what an AI system produces before anything reaches a customer or a supplier, or changes a record in your ERP, the system that holds your orders and prices. This guide shows which actions at a manufacturer or distributor need that approval, and how to design the review so people read what they approve. The last sections cover what Canadian rules and standards say.
What human in the loop means in AI
Human in the loop AI, often shortened to HITL, describes a setup where the AI proposes and a person decides. The AI drafts a quote, a reply or a change to a record. A named person approves it, edits it or rejects it, and only then does anything happen. Each edit and each rejection also shows where the AI went wrong, so the setup behind it can be corrected.
NIST, the US standards agency, describes the range in its AI Risk Management Framework (January 2023): “Human-AI configurations can span from fully autonomous to fully manual.” A system can decide on its own, “defer decision making to a human expert,” or serve a person “as an additional opinion.” NIST adds that some systems “may not require human oversight,” such as models “used to improve video compression.”
Two related terms describe the other points on that range. With a human on the loop, the AI acts on its own, and a person watches the results and can step in afterward. With a human out of the loop, nobody reviews. For actions that commit your company to something, the person should decide before the action happens.
AI agents are software that uses an AI model to take actions, such as sending an email or writing to a database. Some tools for building them now offer this pause. LangChain, a software framework for building agents, documents it: its human-in-the-loop feature checks each action the model proposes against a policy you set. When an action needs review, the agent stops before that action runs and waits for a person.
Human in the loop in machine learning
In machine learning the term has a narrower meaning. People label the examples a model learns from, and they review the predictions the model is least sure of. Amazon Augmented AI (A2I), a service from Amazon Web Services (AWS) for human review, sends human reviewers “low-confidence predictions or random prediction samples.” AWS says A2I is no longer open to new customers. Its two ideas carry over to the business approvals below: send a person the cases the system is least sure of, and re-check a random sample of the rest.
Which actions need a person’s approval at a plant or distributor
A simple rule covers most cases. A person approves anything that leaves the building, commits money, changes a record other people rely on, or is hard to undo. Answers that stay inside the company and change nothing can run without approval, as long as someone checks a sample. For the jobs AI does in a plant, see AI for manufacturing.
OWASP, a non-profit foundation that publishes security guidance, calls the opposite risk excessive agency: an AI tool with more functions, permissions or autonomy than its job needs. One of its fixes is to “require a human to approve high-impact actions before they are taken.” AI security covers what an AI tool can reach, and secure AI at work covers the account settings.
| Action | What goes wrong | Who approves first | Extra checks |
|---|---|---|---|
| Customer quote | A wrong price, quantity or lead time commits you in writing. | The estimator, or the salesperson who owns the account | Read back the price, quantity and ship date. Send the quote to a second approver when its margin is below your margin floor, the lowest margin you accept, or its total is above a set amount. |
| Supplier purchase order | Money is committed to the wrong part, quantity or supplier. | The buyer | Compare it with the requisition and the last price paid. Send it to a second approver when it is above your signing limit or goes to a new supplier. |
| Customer email | A promise in writing the plant cannot keep, or a message to the wrong person. | The person whose name is on the email | Show the recipient and every number or date in the message at the top of the review screen. |
| ERP change to a price, a bill of materials or a customer record | One bad value spreads to every order that uses it. | The person who owns that record | Show the value before and after, and the open orders it affects. |
| Internal answer from your records | Someone acts on a wrong number. | No approval step, since nothing leaves the company or changes | Show the source record behind each number. Re-check a random sample every week. |
In Moffatt v. Air Canada (February 14, 2024), British Columbia’s Civil Resolution Tribunal decided a small claim about a refund for a bereavement fare. The customer had used the chatbot on Air Canada’s website while researching flights. The tribunal wrote: “It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot.” It ordered Air Canada to pay $812.02, including $650.88 in damages. It is one small-claims decision, and it treated what a chatbot told a customer as the company’s own information.
Map the approvals for one job
Tell Derik which job you want AI to help with, such as quoting or order entry. He will tell you where a person should approve and what the review screen should show.
Start a conversationWhy approvals turn into rubber stamps
An approval step helps only when the approver reads what they approve. The risk has a name: automation bias. NIST’s Generative AI Profile (July 2024) warns that people “may over-rely on GAI systems,” meaning generative AI, and calls this “automation bias, or excessive deference to automated systems.” Article 14 of the EU AI Act requires that the people who oversee a high-risk AI system be able to remain “aware of the possible tendency of automatically relying or over-relying on the output.”
Experienced people are affected too. In a 2023 study published in Radiology, 27 radiologists read mammograms with suggestions presented as coming from an AI system, and some of the suggestions were deliberately wrong. The RSNA’s summary reports that inexperienced readers were right in almost 80% of cases when the suggestion was correct, and in less than 20% when it was wrong. Radiologists with more than 15 years of experience on average fell from 82% to 45.5%.
Software developers show the same pattern with AI coding tools. Anthropic reported in March 2026 that users of Claude Code, its AI coding tool, approve 93% of permission prompts, the requests the tool shows before it runs a command or changes a file. It says this “leads to approval fatigue, where people stop paying close attention to what they’re approving.” In August 2026 it reported a 97% approval rate. It also reviewed sessions that its safety systems had flagged between May and June 2026, drawn from accounts that had opted in to data use. Among those flagged sessions, 6.3% of the ones that went through manual approval contained a harmful action the user had not explicitly asked for, at severity 7 or higher on a scale of 0 to 10. Anthropic’s example of severity 7 is deleting rows from a production database, the live database a business runs on. For sessions in auto mode, where automated checks decide which commands can run without asking the user, the figure was 2.4%. These figures come from developers approving a coding tool’s commands, so read them as a warning sign for business approvals.
How to design an approval step people actually read
Each control below either narrows what the reviewer has to check or measures whether the review works. Start with read-back and the audit log on the job that carries the most money, then add the others.
Read back the numbers
Pilots do this with air traffic control. The Aeronautical Information Manual of the US Federal Aviation Administration asks pilots to read back altitude assignments, vectors (headings) and runway assignments, because “the read back of the ‘numbers’ serves as a double check between pilots and controllers.” Put the numbers from each draft on one line at the top of the review screen: price, quantity, unit, ship date and recipient. For a large order, ask the approver to type the quantity or the total before the approve button works.
Show the change before and after
For a write to the ERP, show the current value, the proposed value and what the change touches, such as the open orders that use the part. The Canadian Centre for Cyber Security (the Cyber Centre), in its AI security primer, lists this step among its defences against prompt injection, meaning instructions hidden in the text an AI reads: “Validate downstream actions (files, code and tools) before execution.” A before-and-after view is how a person does that for one record.
Route by threshold
Send each draft to the level of review its risk calls for. Apply the idea behind A2I’s review of low-confidence predictions to business risk. A quote below your margin floor, an order above your signing limit, a new customer, a part you have never made, or a price change above a set percentage goes to a second approver. Routine drafts stay with one approver, so the second approver sees only the drafts that need a closer look.
The EU AI Act uses a two-person rule for one narrow case, remote biometric identification, where a match must be “separately verified and confirmed by at least two natural persons.” For a smaller company, a similar rule is a second signature on purchase orders and quotes above a set amount.
Re-check a random sample
An approval record only shows that someone clicked approve, so a second check is how you find what slipped through. Each week, pull a random sample of approved items, for example ten quotes and ten purchase orders, and have a second person check them against the source documents. A2I uses random samples for the same reason, to “audit its predictions on an ongoing basis.” The share of errors the second check finds estimates what the approvals missed.
Track the override rate
Record every edit and every rejection with a short reason. NIST’s framework points the same way: “Data about the frequency and rationale with which humans overrule AI system output in deployed systems may be useful to collect and analyze.” An approval rate close to 100% for months can mean the AI is accurate, or it can mean approvers have stopped reading. The random sample tells you which.
Give the approver more than yes or no
Offer approve, edit and reject. LangChain’s pause offers these three, plus a reply to the agent in place of the action. With an edit option, an approver can fix a nearly right draft in place and then approve it. A rejection with a reason goes back to whoever maintains the system.
Keep a stop control
Someone must be able to pause the AI at once, for one workflow or for all of them. NIST’s MANAGE 2.4 calls for mechanisms “to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use.” Article 14 of the EU AI Act describes a “‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state.” In practice, a stopped system leaves its drafts in the queue and sends nothing.
Keep an audit log
Record who approved what and when, the version they saw, the source records behind it, what they changed, and what was finally sent or written. Action 9 of the Cyber Centre primer asks for “auditable decision trails.” For decisions about a person, Quebec’s private-sector privacy act adds a retention rule in section 11: the information used “is kept for at least one year following the decision.”
What Canadian rules and standards say
The texts below deal mostly with decisions about people and with guidance for organizations that use AI. This section is not legal advice, and AI governance covers the wider legal picture.
Quebec: Law 25 and automated decisions
Section 12.1 of Quebec’s private-sector privacy act, added by Law 25, applies to any enterprise that “uses personal information to render a decision based exclusively on an automated processing of such information.” The enterprise must tell the person no later than when it informs them of the decision. On request, it must tell them which personal information was used, “the reasons and the principal factors and parameters that led to the decision,” and that they have the right to have the information corrected. The person “must be given the opportunity to submit observations to a member of the personnel of the enterprise who is in a position to review the decision.”
The Commission d’accès à l’information (CAI), Quebec’s privacy regulator, lists this notice among the main changes of September 2023. Failing to give the notice, or the chance to submit observations, can bring a monetary administrative penalty under section 90.1. For an enterprise, section 90.12 sets the maximum at “$10,000,000 or, if greater, the amount corresponding to 2% of worldwide turnover” for the preceding fiscal year.
Section 12.1 covers decisions about a natural person made only by automated processing of personal information. Whether a given workflow falls under it, for example a quote to a sole proprietor, and whether a person’s review changes the answer, are questions for privacy counsel.
Federal: Bill C-36 at second reading
Bill C-36, introduced on June 15, 2026, would enact the Protecting Privacy and Consumer Data Act. On September 27, 2026, LEGISinfo showed it at second reading in the House of Commons. Its definition of an automated decision system is broad: “any technology that assists or replaces the judgment of human decision-makers” through a technique such as a rules-based system or machine learning. If an organization used an automated decision system to make a prediction, recommendation or decision about a person that “could have a legal or similarly significant effect” on them, it would have to explain it on request. The person could make written representations to an employee “who is able to review the prediction, recommendation or decision.”
Our reading: because the definition covers technology that assists a decision-maker, a system can count as an automated decision system under the bill even when a person approves each output. The earlier Artificial Intelligence and Data Act was part of Bill C-27, which was still in committee when Parliament’s session ended on January 6, 2025.
Cyber Centre and Treasury Board guidance
The Cyber Centre’s Top 10 AI security actions (ITSAP.10.049), published in May 2026, is guidance. Action 9 reads “Ensure that human-in-the-loop oversight and execution controls are in place.” Its measures include “Embed human checkpoints in automated and multi-agent workflows” and kill switches “for high-impact actions.” Action 10 adds “human review for critical outputs.” The primer’s example is a 2025 case in which an HR technology company faced “reputational damage and legal scrutiny” after it removed human review from AI candidate screening.
The Treasury Board’s Directive on Automated Decision-Making applies only to federal institutions, but its impact levels make a useful model. At levels I and II, “The system may make decisions and assessments without direct human involvement.” At levels III and IV, “The final decision must be made by a human.” A manufacturer can sort its own AI actions the same way. Low-impact answers run and get sampled, and anything that commits money or reaches a customer waits for a person.
NIST AI RMF and the EU AI Act
NIST says its framework “is intended to be voluntary.” Two of its outcomes bear directly on approvals. GOVERN 3.2 reads: “Policies and procedures are in place to define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems.” MAP 3.5 reads: “Processes for human oversight are defined, assessed, and documented in accordance with organizational policies from the GOVERN function.” In plain words, write down who approves what, and check that it happens.
Article 14 of the EU AI Act sets human oversight duties for high-risk AI systems. Under Article 2, the Act can reach a Canadian company that places AI systems on the EU market, or whose AI output is used in the EU. A European Parliament briefing from June 2026 reports that EU lawmakers agreed to set fixed dates for the high-risk rules: “2 December 2027 for stand-alone high-risk AI systems, and 2 August 2028 for high-risk AI systems embedded in products.” The same agreement removed AI-enabled machinery products from the Act’s direct high-risk regime and left them to the Machinery Regulation. The final act, Regulation (EU) 2026/1744, was published in the EU’s Official Journal on July 24, 2026, according to the European Parliament’s Legislative Observatory. AI governance covers who the Act reaches.
How ThriveAI helps
ThriveAI is an AI engineering company in Ottawa that builds private AI systems for manufacturers and distributors in Ontario and Quebec, on their own data. Derik Lawlis, the founder, leads every project and stays close to the build.
In every system ThriveAI builds, a named person approves each quote and message before it is sent, and each change before it is written back to your ERP. Nothing is sent without that approval. Every answer shows the record it came from, and the log of drafts, approvals and edits is stored with your data.
The platform ThriveAI builds on is designed to keep your data on its own server in Canada that only your company uses. You choose the model that reads it. One option is an open-weight model running on that server. An open-weight model is one whose files are published, so you can run it on your own hardware. The other is a hosted model under a written zero data retention agreement, a contract under which the provider does not store your requests or its answers after it responds, except where the law requires it or its safety systems flag a request (Anthropic’s data retention page, read September 27, 2026). The agreement names the models it covers, and a hosted model may process requests outside Canada. Private AI for business, sovereign AI and local LLMs explain the choices, and Claude API pricing covers what a hosted model costs to run. For how the pieces fit together across a company, see enterprise AI platforms.
Working sessions run on site with your team, in French or English. ThriveAI also runs hands-on AI training on your own documents. Stage one is a working prototype for one job, built in weeks, and you keep it. For the company and how a project runs, see About ThriveAI.