Research · Brief
Fail toward a human, never toward an action
Every system fails eventually. What matters is which way it falls. This brief sets out the failure modes of an agent that acts, and argues that each one should end with work waiting for a person, never with an action slipping through.
- Author
- elan1 research team · Research
- Published
- Reading time
- 3 min read
Engineers talk about systems that fail open or fail closed. A door that fails open lets everyone through when the power goes; one that fails closed keeps everyone out. For agents that change records, the equivalent question is whether a failure produces an action or a pause. Our answer is a rule: fail toward a human, never toward an action.
Five ways an agent system fails
- The model is unsure. Confidence is below the signed line, or inputs are missing.
- The model is wrong but confident. Calibration has drifted or the case is unlike anything it has seen.
- A dependency is down. The model endpoint, the rules engine or the system of record is unreachable.
- The approver is absent. A request waits and nobody answers.
- The rules conflict or do not cover the case. Nobody wrote a rule for this action type.
Which way each one falls
For each failure, the safe direction is the same: the action does not run, and the work waits where a person can see it.
- Unsure: the case is routed to review or hold, never acted on.
- Confidently wrong: limits still apply. A payment above the threshold waits for a person whatever the model's confidence, and bank-detail changes are refused by default.
- Dependency down: actions stop; they do not slip through. In elan1 Govern, an error refuses.
- Approver absent: the request expires after 24 hours and is refused. Nothing proceeds by default.
- No matching rule: unmatched calls are refused. A new action type cannot run because nobody thought of it.
The cost of failing safe
Failing toward a person is not free. Queues build when a dependency is down, and approvers get more requests when calibration drifts. That cost is visible and recoverable: work waits and is then done. The cost of failing toward an action is often neither: a payment released in error may never come back.
The record makes the trade-off manageable. Because every refusal, expiry and hold is written down with its reason, operations teams can see which failure is generating the queue and fix the cause rather than loosening the rule.
The emergency stop
Sometimes the right response is to stop everything. elan1 Govern is designed with one emergency stop that pauses every agent at once and starts the incident report. RBI's draft model-risk guidance asks for a kill switch for exactly this reason, and CERT-In's directions require cyber incidents to be reported within 6 hours, which is far easier when the stop and the record are the same system.
Checklist
- Every action type has a rule; anything else is refused.
- Every error path ends in refusal, not retry-and-act.
- Every approval has a deputy and an expiry.
- One person can stop all agents, and knows how.
- Every refusal, expiry and hold is recorded with its reason.
Key takeaways
- Agent systems should fail toward a person: when anything goes wrong, the action does not run and the work waits.
- Errors, unmatched calls and unanswered requests are refused, never allowed by default.
- An emergency stop and a complete record turn an incident into a recoverable queue rather than an irreversible loss.
Further reading
Product capabilities named here carry their current status (available, in pilot or designed) on each product page and in Trust and security. Figures marked “Design target” are specifications, not measurements.
About the author
elan1 research team
Research
The research team writes elan1's papers and briefs. Each one states its method and its limits, labels design targets as targets, and cites third-party figures to their primary source. elan1 has no customer results to publish yet, and our research does not imply any.
