AI can produce an answer in seconds.
The answer may be clear, plausible, and even correct. Yet something about it can still feel wrong.
I do not think this discomfort comes only from hallucinations or poor model accuracy. Sometimes the real problem is simpler:
The AI returned an output, but it did not return the work in a form that another person can safely continue.
This is not a new problem created by AI. It is the same problem we already have when delegating work to another person.
What do we expect when we delegate work?
Imagine a manager asking a team member:
Please prepare a proposal for reducing next month's operating costs.
The team member reviews several documents, compares multiple options, and replies:
We should choose Option A.
The requested conclusion has been delivered. But has the work really been handed back?
The manager still does not know:
- What objective the team member optimized for
- Which documents and facts were examined
- Which assumptions and constraints were used
- Which alternatives were compared
- Why Option A was preferred
- Which conditions remain unverified
- What must be reconsidered if the situation changes
The original request may not have explicitly demanded all of this. Even so, we normally expect a competent team member to understand the purpose of the assignment and to return enough information for someone else to review, approve, revise, and continue the work.
That information is not additional reporting attached to the work.
It is part of the handoff condition that makes delegation possible.
AI often returns the conclusion without the handoff
Now replace the team member with an AI assistant.
The AI immediately recommends Option A and produces a polished explanation. Because the answer arrives so quickly and looks complete, it is easy to confuse the existence of an output with the completion of the work.
But the same questions remain:
- How did the AI interpret the objective?
- What was considered in scope and out of scope?
- Which sources were actually used?
- Which assumptions were supplied by the user, and which were introduced by the AI?
- What alternatives were rejected?
- Which evaluation criteria determined the recommendation?
- What is uncertain or still missing?
- What should a human verify next?
If these questions cannot be answered, the human receiving the output cannot take responsibility for it.
The output may be correct, but it is not yet transferable as work.
A software example
Suppose an AI coding agent is asked to fix a bug.
It edits several files, the tests pass, and the application appears to work. Still, the reviewer may feel uncomfortable accepting the change.
The discomfort may not come from the code itself. It may come from not knowing:
- Which requirement the agent treated as authoritative
- Which existing behavior it intended to preserve
- Whether it found and followed the project's design rules
- Which assumptions it made about unspecified behavior
- What tests it ran and what those tests actually covered
- Whether related code paths remain unverified
- Whether the change is a local repair or an architectural decision
“The tests passed” is a result. It is not a complete handoff.
If the next developer must rediscover all of the context before reviewing or modifying the change, the work was not transferred. Only the generated artifact was transferred.
This is not a request for hidden chain of thought
At this point, it is easy to misunderstand the argument.
I am not asking an AI system to reveal its private chain of thought. A generated explanation of “what the model was thinking” may itself be a post-hoc story. More explanation can also make an answer more persuasive without making it easier to verify.
What we need is not an imitation of internal thought. We need observable and verifiable work information:
- The objective and scope
- The facts and sources used
- The assumptions and constraints
- The alternatives and evaluation criteria
- The actions performed and validation results
- The unresolved questions, risks, and uncertainty
- The resulting artifact and the next required action
These are external properties of the work. A reviewer can inspect them, challenge them, and update them when conditions change.
Explainability is not enough
Much of the discussion around trustworthy AI focuses on explainability. NIST, for example, describes four principles for explainable AI: providing reasons or evidence, making explanations meaningful to the intended user, ensuring that explanations accurately reflect the system, and recognizing the system's knowledge limits.
These principles are important, but workplace delegation requires something broader than an explanation of an output.
The receiver must also understand the status of the work:
- What has been completed?
- What has been checked?
- What has not been checked?
- Which decisions have been made?
- Who owns the next decision?
This is closer to a handoff problem than a pure explanation problem.
Research on human-AI decision-making uses the term appropriate reliance. The goal is not to make people trust AI more. The goal is to help people accept correct AI advice and reject incorrect advice.
That distinction matters. A polished explanation may increase trust while doing little to help a person distinguish a correct recommendation from an incorrect one.
The practical question is therefore not:
Does this answer sound convincing?
It is:
Has enough verifiable information been transferred for me to decide whether to rely on it?
Reproducibility in non-routine work
In routine work, reproducibility often means that different people following the same procedure produce the same result.
That definition does not fully apply to management decisions, investigation, system design, or review. Conditions change. Assumptions change. New facts appear. Two competent people may reasonably reach different conclusions.
For this kind of work, what should be reproducible is not necessarily the conclusion. It is the ability to reconstruct and continue the work:
- Understand the original purpose and situation
- Confirm the facts and assumptions
- Re-evaluate the alternatives using explicit criteria
- Identify what must change when conditions change
- Preserve unresolved issues for the next person
- Continue from the current state instead of starting again
This is the form of reproducibility required for a reliable handoff.
AI does not create a new delegation problem
AI makes the old problem easier to ignore.
A human colleague usually needs time to investigate and produce a result. During that time, there are opportunities to ask questions, discuss assumptions, review intermediate findings, and correct misunderstandings.
AI compresses that process into seconds. The intermediate coordination disappears, while the final output looks finished.
As a result, we may receive an answer before we have established the shared context needed to evaluate it.
This is why increasing model intelligence alone may not remove the discomfort. A more capable model can produce a better answer, but if the work arrives without its purpose, assumptions, evidence, validation state, and unresolved questions, the receiver still cannot safely own it.
The real expectation
We often say that we want AI to work like a capable team member.
That does not mean treating AI as a person. It means applying the same conditions required whenever work is delegated:
Do not return only the result. Return the work in a state that another person can understand, verify, revise, and continue.
The discomfort we feel toward AI output may therefore be an important signal. It may indicate not that the answer is wrong, but that the handoff is incomplete.
If an AI gives you the correct answer but leaves you unable to verify, revise, or hand off the work, has the work actually been completed?
References
- NIST, Four Principles of Explainable Artificial Intelligence
- Schemmer et al., Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making
- Schoeffer, De-Arteaga, and Kuehl, Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
- Müller et al., Impact of the communication and patient hand-off tool SBAR on patient safety: a systematic review
Top comments (0)