Recommend only: where AI stops in clinical intake

elan1 Decide is built to make millions of everyday decisions quickly, with a stated confidence and a full record. For most of them, such as routing an invoice exception or classifying a support ticket, the routine cases can be automated once confidence clears a threshold that your risk owner signs.

For a short list of domains, we made a different choice. Credit, anti-money-laundering outcomes, clinical intake and hiring are always recommend-only. elan1 Decide can rank, summarise and suggest. A person decides.

Why confidence is not enough

A calibrated model can be right 98 times in 100 and still be the wrong tool to decide alone. In clinical intake the cost of the other two is not symmetrical: sending a stable patient to a senior clinician wastes some time, but under-triaging a patient with an atypical presentation can cause harm that no record can undo.

The cases a model is least sure about are also the ones where the people affected are least like the data it learned from. Automating the confident majority and routing the rest sounds sensible until you notice that the rest is where the risk concentrates.

What recommend-only means in practice

• Decide reads the intake form and notes and returns a suggested priority with its confidence and the evidence it used.
• The suggestion is shown to the intake nurse or clinician, never sent onward on its own.
• The person accepts, changes or rejects it, and their decision is what reaches the hospital system, through elan1 Govern.
• Both the suggestion and the human decision are recorded, so disagreement between them can be studied.

That last point matters. Over time, the record shows where the model and the clinicians disagree. That is the most useful input a clinical governance committee can get, and it never requires the model to act alone.

What the rules already say

Regulation points the same way. The EU AI Act treats AI that is a safety component of a medical device, or that is used to evaluate and classify emergency calls or to triage patients in emergency care, as high-risk, with duties of human oversight and logging. In India, the Digital Personal Data Protection Act, 2023 governs how patient data is used, and consent and purpose limits apply to every model that reads it.

Where automation does belong in a hospital

None of this means healthcare should avoid agents. The administrative work around care is enormous: admissions paperwork, billing queries, pharmacy stock, insurer correspondence. Those processes can run with agents today, with every action that touches patient data allowed, sent to a named person, or refused, and recorded.

The line we draw is simple to remember. Agents can carry the work around a clinical decision. They do not make it.