The useful question when you design with AI isn't whether to have a human in the loop. It's where the authority boundary sits, and that is a decision about consequences, not capability alone.
Two systems I'm building right now, both AI-heavy, with different authority boundaries. In one, AI writes database migrations but never runs them. A migration can change production schema or data in ways that are hard to reverse. The AI writes the SQL; it does not execute. I review and decide whether it runs. In the other, items the AI generates and self-rates with low confidence route to a human for review. Most items ship. A small fraction land in front of me.
Different gate placements. Same architect's question: where does AI authority end and human authority begin? Both answers come from the consequence of a wrong action, not from how clever the model is.
In the loop, on the loop, responsible for the loop
Not every "automated" system puts a human in the loop per action. Payment systems auto-decline. Deployment systems auto-rollback. Monitoring auto-restarts. In those cases humans aren't gating individual decisions. But humans wrote the rules, tuned the thresholds, and watch for drift. There is still a human somewhere. The human is on the loop or responsible for it, not in it.
This piece is about where the human belongs: in the loop, on the loop, or responsible for it. The wrong boundary either hides staff cost in a demo or lets consequential actions ship without any gate at all.
What drives the gate placement
Two questions, both carried over from an earlier piece in this series. How big is the blast radius if the action goes wrong. How reversible is it.
A migration can change production schema or data in ways that are hard to reverse. The blast radius can be large and the recovery path is slow and lossy. That action goes behind a mandatory human gate.
A generated item routed to a review queue has a small blast radius per item (one wrong classification, one bad draft) and is held before release. The gate is conditional: the AI acts most of the time; a human sees the uncertain cases.
An auto-decline on a transaction with a high fraud score has a contained blast radius per decision and a well-understood error path (customer retries, support un-declines, risk team retunes). That action can be autonomous. Note that this gate type is not AI-specific; rule-based automation lives here too.
The architect's job is to look at each action, assess the blast radius and reversibility, and pick the gate.
Three gate types
Mandatory human approval. The AI prepares the work but cannot act on it. A qualified human reviews and executes. Used for actions with large blast radius and slow-enough reversibility that per-item review is affordable. Pre-merge code review and privileged database operations are the canonical cases.
Conditional routing. The AI acts on most items; exceptions route to a human. The trigger can be an AI confidence score, a deterministic business rule (value out of range, regulatory flag), or a webhook event. Used for high-volume actions where most items are safe to automate but a tail needs judgment. Three common variants:
Confidence-triage. The AI self-rates each item; items below a threshold route to a human. Caveat: self-reported confidence is not a reliable safety signal on its own. A confidently wrong model will not route itself. This pattern works best when stakes per item are bounded, or when confidence is combined with other risk signals.
Escalation rules. Specific conditions trigger a person who decides what to do. Code handles detection. The person responds. The deterministic trigger is what makes this more trustworthy than confidence-routing alone.
Notification-plus-judgment. A webhook fires on a consequential event. The handler logs it and notifies the designated human. It does not auto-act. The subscription dispute case I worked through earlier in the series is one of these.
Autonomous within a reversible, bounded scope. The AI (or the rule-based system) acts; no human reviews each action. Used when the blast radius is small, the action is cheaply reversible or recoverable, and the system monitors for drift. Auto-rollback on failing health checks, auto-decline on a high fraud score, auto-retry on transient errors. This is well-governed autonomy by design, not autonomy by default.
A real system usually has all three. The architect's work is deciding which action belongs in which gate.
How this scales
Early in a system's life, the mandatory gate is often one person, usually the person who built it. Efficient when volume is small and hard cases are rare.
The confidence-triage system I'm building has me as the sole reviewer today. At production volume it would need tiered review (routine reviewers for the clear cases, senior reviewers for the hard ones), random sampling of the items that shipped without review (confident wrongness has to be caught somewhere), and routing by expertise. The migration gate stays mandatory even then. Different gate types scale differently.
The fundamental principle survives scale: automation handles the work it is authorized to do; qualified humans retain the decisions and exceptions that require their judgment. The implementation evolves.
In pre-launch solo builds, the architect and the operator are the same person. At production scale they are different jobs with different headcount. One designs the gates; others work them.
When someone pitches "autonomous"
Ask what happens in the hardest 5% of cases. Not the demo cases. The actual tails.
If the answer involves a human looking at something, the system has a hidden human-in-the-loop. Ask how it scales. A good answer names a tiered structure and routing logic, not one hero on call.
If the answer involves the agent handling it alone, ask who notices when it is wrong and how fast. "We check the logs weekly" is unmonitored autonomy. That is the absence of a gate, not gate type three.
If the answer explains which actions are autonomous because their blast radius is bounded, which are gated because they change production state, and which land in a queue for human review, you have found an architect at work.
The architect's job
As models get better, the human's job in these systems shifts. Less time spent reviewing routine outputs. More time spent deciding where authority boundaries belong, and watching for cases where the boundaries drawn earlier no longer match the consequences.
The architect's job is not to decide what the AI is capable of doing. It is to decide what the AI should be authorized to do, what must be reviewed, and what happens when the system gets it wrong.
The next time someone says "fully autonomous," ask where the authority boundary sits. If the answer is "the AI is good enough now," they are describing capability. The question is architecture.