human in the loop AI7 min read · 06 / 20

The best AI decision system I designed did not make the decision.

I thought the breakthrough would be more autonomy. The real value appeared when AI removed the confusion surrounding a decision and left the judgement with the person responsible.

Written by Mitesh DabhiAI Adoption Specialist
Hand-drawn AI system organizing evidence and highlighting risks while a human makes the final decision.
AI prepared the evidence, surfaced uncertainty and made the decision easier without pretending to own it.

The system could recommend an answer. The harder question was whether it should be allowed to decide.

We were reviewing a workflow where a manager had to assess risk. The AI could summarize recent activity, highlight delays and suggest a response. It was easy to imagine one more step: let the system take the action automatically.

That possibility felt exciting and wrong at the same time. The model had made the preparation faster, but it did not carry the accountability, relationship history or consequences of a bad call.

I realized we were measuring the system by how much human involvement it removed instead of how much better it made the human decision.

01 / The work before judgement

Most of the time was being lost before the decision, not during it.

The manager searched for context, compared sources, reconstructed history and identified what was missing. That preparation took longer than choosing the next action.

  • Summarize recent activity
  • Highlight missed deadlines
  • Surface unresolved dependencies
  • Extract concerns
  • Compare progress with plan
  • Suggest questions to investigate

AI was unusually good at organizing that evidence. The opportunity was not to replace the manager. It was to make sure the manager began with clarity rather than scattered information.

02 / The human boundary

I separated preparation from consequential action.

Drafting, classifying and recommending could happen automatically. Sending external communication, changing a financial record or rejecting an important request required a responsible person. The boundary became part of the product, not an afterthought.

The operating model

I route work by confidence, consequence and reversibility instead of applying one level of autonomy everywhere.

01

Prepare

Let AI gather, summarize and structure the evidence a person would otherwise spend time assembling.

Context · history · risks · missing data
02

Review

Route ambiguous, medium-confidence or meaningful cases to someone who can correct the output.

Edit · approve · reject · comment
03

Escalate

Stop automatic action when the risk is high, the information conflicts or the outcome is difficult to reverse.

Legal · financial · employment · external
03 / Review as learning

Every correction showed us how the workflow could become better.

When a reviewer edited the output, the change revealed missing context, weak source data, tone mismatches or unclear instructions. Review was producing product intelligence.

01Decision owner02Evidence required03Confidence rule04Approval point05Escalation path06Reversible actions07Review history08Feedback capture

We could improve the prompt, retrieval, validation and interface because the human step made failure visible instead of allowing it to travel downstream.

The principle I use“Autonomy is not the goal when judgement matters. Clarity, speed and accountable action are the goal.”
04 / A different measure of value

Eighty percent of the preparation created more value than risky full autonomy.

The manager still made the call, but reached it faster with a clearer view of the evidence. The system reduced cognitive load without reducing responsibility.

Faster preparationClear evidenceVisible uncertaintyHuman accountabilitySafer actionBetter feedback

That balance increased trust. People were more willing to use AI when they understood what it prepared, what it did not know and where they remained in control.

What I carry forward

I design AI around the burden surrounding a decision before I automate the decision itself.

The most valuable system may never make the final call. It can organize the evidence, surface the risk and identify what is missing.

When judgement begins with clarity, people decide faster and with more confidence, and the technology earns a responsible place in the workflow.

AI should not remove human judgement where judgement matters. It should remove the confusion that makes good judgement harder.