The triage problem
Banks, NBFCs and payment companies in India must monitor transactions and report suspicious ones to the Financial Intelligence Unit, FIU-IND, under the Prevention of Money-laundering Act, 2002 and its rules. Most monitoring still starts with rules: thresholds, velocity checks and typology scenarios. Rules are explainable and easy to audit, but they are blunt, so they generate many alerts that turn out, on review, to be ordinary activity.
Every alert needs an analyst to gather the customer's profile, recent transactions and any adverse information, then decide whether to close it, escalate it or file a report. The work is repetitive, time-bound and error-prone at volume. KYC review has the same shape: periodic updates of customer information, re-verification and checks against lists, many of them routine and a few that matter a great deal.
The FATF's 2021 report on new technologies for AML/CFT describes how AI and analytics can make this work more accurate and better targeted, and it is equally clear that human review remains essential. The task is to use AI to put analyst time where the risk is, without letting it decide what only an accountable person should.
Where AI fits: scoring, summarising, routing, not closing
| Task | AI's role | Person's role |
|---|---|---|
| Alert scoring | Estimate how likely an alert is to need escalation | Set and review the thresholds |
| Case preparation | Gather profile, transactions, linked accounts and adverse media into a summary | Check the summary against source records |
| Routing | Send each case to the right queue by score and risk tier | Own the routing policy |
| Disposition | Draft a closure or escalation note with reasons | Decide to close, escalate or file |
| Reporting | Prepare a draft suspicious transaction report | The Principal Officer decides whether to file |
The line is simple: AI may prepare, a person decides. Even where your policy lets low-risk, high-confidence alerts be closed with light review, a named analyst should own each closure, and a sample should be re-reviewed. A regulator asking why an alert was closed expects to hear about a person's decision, supported by the evidence the AI gathered.
Keep the data the model sees to what the decision needs. An alert-triage model rarely needs a customer's full document images; a case summary rarely needs every field in the core banking record. Minimising inputs reduces the personal data processed under the DPDP Act, narrows what can leak, and makes the model's reasoning easier to explain.
Calibrated confidence: a raw score is not a probability
Most models output a number between 0 and 1. It is tempting to read 0.9 as “90% likely”. It usually is not. A well-known study by Guo and colleagues (2017) showed that modern neural networks are often overconfident: their stated confidence is higher than their actual accuracy. Large language models asked to rate their own certainty can be worse.
Calibration fixes this. A model is calibrated when, among all cases it scores at 0.9, about 90% turn out to be true positives. You calibrate on your own labelled cases, from your own customers and typologies, by comparing scores with outcomes and fitting a correction. A reliability diagram, which plots predicted confidence against observed frequency, shows at a glance whether the model is over- or under-confident in each band.
Calibrate before automating anything, and re-check regularly. Customer behaviour, fraud patterns and the model's inputs drift, and a model that was calibrated last year may not be now. A route that depends on a confidence threshold is only as good as the calibration behind it.
Routing by risk: four routes
Combine two inputs: the calibrated score, and a risk tier your policy assigns from the customer, the product and the typology. Then give each case one of four routes.
- Prepare automatically. Low-risk tier and high calibrated confidence that no escalation is needed. The AI prepares the case and the closure note; an analyst confirms in bulk, and a sample is re-reviewed.
- Analyst review. Anything below the confidence threshold, or in a medium-risk tier. The analyst sees the case summary and the score side by side.
- Second pass. Conflicting signals, such as a low score but a new adverse media hit. A more thorough check runs, by a stronger model or a senior analyst, before routing again.
- Hold for a person. High-risk tiers, politically exposed persons, sanctions near-matches, or missing data. A senior analyst decides; nothing is prepared to close.
Write the thresholds down, version them and record which version applied to each case. When you change a threshold, you should be able to show what it would have done to last month's cases.
KYC review: documents, periodic updates and adverse media, with a checker
RBI's Master Direction on KYC requires customer due diligence at onboarding and periodic updation of KYC by risk: at least once every two years for high-risk customers, every eight years for medium-risk and every ten years for low-risk. Much of that work is routine: checking that documents are valid and consistent, confirming that nothing has changed, and screening names against lists and adverse media.
AI can read documents, extract and compare fields, flag inconsistencies, summarise adverse media and draft the reviewer's note. The reviewer, the checker, confirms the result. Keep a checker on anything that changes a customer's risk rating, restricts an account, or rests on an adverse media match, because these affect customers directly and are the decisions most likely to be questioned.
Periodic updation also has a customer-facing side. The Master Direction asks regulated entities to give customers at least three advance intimations before KYC falls due, including at least one by letter, with clear instructions and the consequences of not updating. AI can draft, personalise and schedule those notices and read the replies, while a person handles the exceptions: customers who dispute a change, cannot provide documents or ask for help.
Explainability and the record regulators expect
RBI's FREE-AI report includes “understandable by design” among its seven Sutras, and its protection and assurance pillars expect explanations, grievance routes and audit. For alert triage, that means each decision should come with reasons a supervisor can follow: which factors raised or lowered the score, what the summary drew on, and what the analyst concluded.
Record, for every case: the alert or review trigger, the inputs used, the model version and calibrated score, the risk tier, the route and the threshold version, the analyst and their decision, the reasons, and timestamps. Link it to the case management system's identifier. This is the record that answers the question every examiner eventually asks: “why was this alert closed, and who decided?”
Prefer structured reasons to free text. A short list of reason codes, such as “salary credit pattern consistent with profile” or “counterparty previously reviewed”, lets you count how often each reason closes an alert, spot reasons that are being overused, and test whether the AI's suggested reasons match the analyst's. Free-text notes are still useful, but they are hard to measure, so keep both.
Measuring it: precision, recall, drift and overrides
Alert reduction is the number vendors like to quote, and on its own it is the wrong one. Closing more alerts faster is only good if the alerts that mattered still reached a person. Measure each route separately:
- Recall of escalations: of the cases that were ultimately escalated or reported, what share did the AI route to review, second pass or hold? This is the safety metric.
- Precision of each route: of the cases routed to review, how many needed it? This is the efficiency metric.
- Below-threshold sampling: re-review a random sample of auto-prepared closures every month, and treat any missed escalation as an incident.
- Calibration drift: compare predicted confidence with outcomes each month, by band.
- Overrides: how often analysts disagree with the AI's suggestion, and in which direction. A rising rate is an early warning.
Report these to the committee that owns the AML programme, not only to the team that runs the model. The NIST AI RMF Measure function describes the same discipline in general terms.
A pilot design: shadow mode first
Do not switch AI triage on and watch the backlog fall. Run it in shadow mode first: the AI scores, summarises and suggests a route for every case, but analysts work exactly as before and do not see the suggestion until they have decided. After a period long enough to include your usual mix of typologies, compare the two.
- Label the shadow-period cases with the analysts' final outcomes, and calibrate the scores against them.
- Measure what each proposed route would have done: how many eventual escalations would have been auto-prepared? The target is none.
- Read a sample of AI case summaries against source records for accuracy and omissions.
- Agree thresholds, routes and the sampling rate with the AML committee and record the approval.
- Switch on one route at a time, usually analyst review with AI summaries first, and automatic preparation last.
The shadow period also gives you the baseline that makes later measurement honest. Without it, any claim about time saved or alerts reduced is a guess.
How elan1 handles this
elan1 Decide is built for this pattern. Each alert or review becomes a typed decision with a tested confidence from the Calibration Lab: no calibration, no automation (see calibrated confidence). The Router can route AI decisions by risk: automatic, human review, a System-2 second pass, or hold. Unsure cases reach the human review queue, where the analyst sees the case and the score side by side.
Decision packs package AML alert triage, KYC review and disputes as ready sets of decisions, routes and review rules. Every answer is kept in the Decision Ledger with its route and reviewer, and any write that follows, such as a change to a customer's risk rating, goes through elan1 Govern's rule check, named approver and proof record. See AML, KYC and disputes for banks and NBFCs.