Fail toward a human, never toward an action

Engineers talk about systems that fail open or fail closed. A door that fails open lets everyone through when the power goes; one that fails closed keeps everyone out. For agents that change records, the equivalent question is whether a failure produces an action or a pause. Our answer is a rule: fail toward a human, never toward an action.

Five ways an agent system fails

1. 01The model is unsure. Confidence is below the signed line, or inputs are missing.
2. 02The model is wrong but confident. Calibration has drifted or the case is unlike anything it has seen.
3. 03A dependency is down. The model endpoint, the rules engine or the system of record is unreachable.
4. 04The approver is absent. A request waits and nobody answers.
5. 05The rules conflict or do not cover the case. Nobody wrote a rule for this action type.

Which way each one falls

For each failure, the safe direction is the same: the action does not run, and the work waits where a person can see it.

• Unsure: the case is routed to review or hold, never acted on.
• Confidently wrong: limits still apply. A payment above the threshold waits for a person whatever the model's confidence, and bank-detail changes are refused by default.
• Dependency down: actions stop; they do not slip through. In elan1 Govern, an error refuses.
• Approver absent: the request expires after 24 hours and is refused. Nothing proceeds by default.
• No matching rule: unmatched calls are refused. A new action type cannot run because nobody thought of it.

The cost of failing safe

Failing toward a person is not free. Queues build when a dependency is down, and approvers get more requests when calibration drifts. That cost is visible and recoverable: work waits and is then done. The cost of failing toward an action is often neither: a payment released in error may never come back.

The record makes the trade-off manageable. Because every refusal, expiry and hold is written down with its reason, operations teams can see which failure is generating the queue and fix the cause rather than loosening the rule.

The emergency stop

Sometimes the right response is to stop everything. elan1 Govern has one emergency stop that pauses every agent at once and starts the incident report. RBI's draft model-risk guidance asks for a kill switch for exactly this reason, and CERT-In's directions require cyber incidents to be reported within 6 hours, which is far easier when the stop and the record are the same system.

Checklist

• Every action type has a rule; anything else is refused.
• Every error path ends in refusal, not retry-and-act.
• Every approval has a deputy and an expiry.
• One person can stop all agents, and knows how.
• Every refusal, expiry and hold is recorded with its reason.