Why an AI risk manager is needed when rules stop working
Hard thresholds hold the load while the transaction flow is homogeneous. As soon as volumes grow and sources diverge — payments in one system, documents in another, client behavior in a third — a rule sees only the part of the picture that was described in advance.
Everything else either slips past or settles in a queue for manual signal review, where an analyst reconstructs context from exports that the alert itself does not contain.
Standard alerts in Grafana honestly show that a threshold has been breached, but do not explain why.
The cost of such a scheme grows nonlinearly: the share of false positives increases, some incidents remain unexplained, and response latency is measured not in minutes but in days.
Risk management turns into processing an accumulated queue rather than controlling transactions at the moment they occur. Each new rule adds an exception, the model becomes murky, and the next process change breaks the settings again.
The AI risk manager removes the limitation differently: instead of enumerating conditions, it builds a behavioral baseline for each entity — client, account, employee, counterparty — and evaluates deviation from it rather than a formal match with a template.
The signal arrives with a significance score and a set of features that influenced it, so the analyst sees the cause, not just the fact.
Thresholds do not disappear — where the regulator requires a hard condition, they remain. But they work in conjunction with the model and enriched context, not instead of it. We explain how such a scheme is assembled and where the boundary between rules and model lies.
What an AI risk manager delivers: response speed and risk coverage
A risk manager based on threshold rules hits a ceiling: thresholds are set with a margin, so the on-call shift drowns in identical notifications, while transaction splitting and correlated events across multiple counterparties slip through. Expanding the list of rules means increasing noise faster than real coverage grows.
The model evaluates an event in context — features, history, deviation from the usual profile — and assigns a priority instead of a binary "triggered." Response speed increases: the review queue no longer depends on peak load, and the time from event to escalation drops from hours to minutes.
The second effect is a unified risk register. Events from different sources are reduced to a common taxonomy (industry classifiers, including MITRE ATT&CK), and each record has an owner, status, and decision history.
This closes a class of failures that rules cannot fix: the same risk lives in two systems under different identifiers, and review proceeds in parallel.
Coverage does not narrow: false positives are reduced through context rather than loosened controls, so the share of confirmed signals in the flow grows.
The effect of implementation is visible in three metrics — response time, share of risks closed before an incident, and number of rules decommissioned. Thresholds and weights are revised based on accumulated statistics, not gut feeling.
Operating modes: rules, model, and hybrid control scheme
We proceed from the fact that the control mode is chosen for regulatory and operational reality, not by the "progressiveness" of the technology.
Rules and thresholds enable fast launch: counterparty, amount, frequency limits and stop lists are described by policy, and every trigger is explainable and reproducible during review.
A trained model covers what cannot be hardcoded into policy — behavioral deviations, atypical transaction sequences, weak signals that individually do not exceed a threshold.
In practice, the choice is not "either-or" but a hybrid mode: rules handle the fast loop and remove routine work from the analyst, the model prioritizes and highlights the non-obvious, and disputed cases go to review under the ISO 31000 procedure.
Comparison of approaches
| Mode | When it fits | Implementation constraints |
|---|---|---|
| Rules and thresholds | Fast launch needed, regulator requirements formalized, few labeled incidents | Does not catch new schemes: circumventing a limit by a percentage below the threshold remains unnoticed |
| Trained model | Accumulated history of incidents and analyst decisions, large transaction flow | Requires labeling and drift control; decisions are not explainable "out of the box" |
| Hybrid scheme | Regulator checks are mandatory, but some risks are behavioral | Traceability required: rule, model, and human write to a single decision log |
How we choose the mode
The choice rests on three things: volume of history, strictness of regulator requirements, and cost of error.
Where the cost of error is high, a rule enforces a hard prohibition and the model only recommends; where catching new patterns matters, the model goes first, and a safeguard rule checks its output.
How we implement: from data audit to handover to operations
Implementing an AI risk manager is a sequence where each step relies on the result of the previous one: from data audit to handover of the system to the client's team. Without this, the model remains a demo rather than a working tool.
- Data audit. We check what can be learned from: history depth, field completeness, gaps, duplicates, and how decision rules changed in the past. The output is a source map and a list of gaps we close before the start.
- Decision scheme. We fix which decisions the system makes, based on which features, and where a human is mandatory. This is the frame within which the model is configured, not a set of wishes.
- Setup and calibration. The model trains on the client's data, and thresholds are tuned to its risk profile. We separately test behavior on rare and deliberately distorted cases.
- Validation on historical data. We run the model over past periods and compare it with decisions actually made: match rate, missed risks, false positives. The output is a report with numbers, not "the model works."
- Pilot launch. The system computes in real time, but a human makes the decision: recommendations go alongside actual review, and it is visible where they diverge. The client determines the pilot duration.
- Handover to operations. We set up quality and data drift monitoring, update policy, access roles, and rollback procedure. The client's team receives documentation and training — after that, the system runs without us.
What the client gets: configurations, schemes, policies
The tangible result is not a presentation or a report on work performed, but a working loop that remains with the client: configuration files, data flow diagrams, policies, and documentation in its repository. Any element can be handed to another team for support without reverse-engineering our decisions.
- Risk rule configurations: trigger thresholds, feature weights, list of exceptions, and rule versions with change history — an edit can be rolled back to the previous set.
- Data flow diagrams: from source to storage, which fields participate in scoring, where personal data is anonymized, and how gaps are handled.
- Access and role policies: who sees the risk score, who confirms a decision, who can change a threshold — with an action log per user.
- Response policy: risk levels, response times for each, escalation procedure, and points where human review is mandatory.
- Disputed case review policy: how a false positive is recorded, who reviews the threshold, and how often rules are revised.
- Documentation: architecture description, data passports, instructions for analysts and the on-call shift, and the procedure for connecting a new source.
- Set of historical data checks: share of covered scenarios, behavior during source failure, and degradation without stopping the process.
Acceptance is based on a checklist: a run on the client's data, comparison of the score with its specialists' decisions, verification of configuration rollback and recovery after source failure.
Case study: how false positives and response time were reduced
Starting point
The client is a payment service where signals about suspicious activity came from six sources: acquiring, fraud prevention, support service, logins to the personal account, account movements, and external lists.
There was no unified client profile — the analyst assembled context for each episode manually from different systems. At peak, this was 1,400 signals per day, about 70% of them went to manual review, and response time for a significant episode stayed in the range of 6–8 hours. Some transactions managed to go through before being blocked during that time.
What was changed
We built a normalization layer: events from all sources were brought to a common schema of "entity — action — context," and risk scoring was calculated for each entity based on a behavioral baseline rather than a one-off feature.
At the same time, events continued to be written to the familiar Elasticsearch storage — the source of truth was not changed, so implementation did not require rebuilding adjacent systems.
Hard stop conditions remained in the rules, and disputed cases went to the escalation queue with context already pulled up.
Metrics before and after
Response time dropped from 6–8 hours to 4–9 minutes: a decision on most episodes is made at input, before the transaction is processed.
The share of automatic decisions grew to 78%, another 12% of signals go to the analyst — these are episodes with incomplete context where hypothesis verification is needed; the rest is filtered out as noise.
False positives fell by a factor of three because recurring legitimate actions stopped being considered deviations.
Signal review no longer required reinforcing the on-call shift on peak days, and a trace is preserved for every denial: which features triggered and at what threshold.
How does scoring by hard rules differ from a trained model?
Hard rules are deterministic logic: a threshold on transaction amount, a limit on counterparty, a combination of conditions, a prohibition when a field is empty.
The verdict is reproducible: the same application always gets the same result, and the reviewer sees which condition triggered.
Decision predictability here is maximal — but rules know only what was put into them and do not account for combinations that were not in the original scheme.
A trained model calculates risk across a set of features and catches non-obvious combinations: a series of small transactions, atypical timing, a mismatch between profile and behavior.
The model's accuracy is higher on a live flow, but it requires labeled data, drifts along with it, and explains an individual decision less well.
| Option | When it fits | Constraints |
|---|---|---|
| Hard rules | Regulatory prohibitions, limits, mandatory checks | Does not see new schemes, number of exceptions grows |
| Trained model | High-volume flow of similar transactions, risk ranking | Requires labeling, drift control, and explanations |
| Combination of approaches | Control with cut-offs and prioritization | Requires a threshold revision policy and discrepancy review |
The combination of approaches is built as follows: rules serve as the framework and hard-cut what is unacceptable, while the model scores the remainder and prioritizes it for manual review.
Every verdict is written to the log together with features and the model version — this preserves decision predictability where it is mandatory. The model's accuracy is measured on a holdout sample, not on the data it was trained on.
Technical concerns: data, false positives, explainability
A risk model is not a ready-made product but a loop: data sources, labeling rules, and feedback from analysts who confirm or reject triggers. Objections from the engineering team are usually the same — dirty data, few confirmed cases, an unexplainable result.
We address each before the start: what exactly we test on your data and where the boundary lies beyond which a human still makes the decision.
There is little data and it is dirty — where to start?
We start with profiling: field completeness, duplicates, discrepancies between systems, share of records without a label. The first loop is rules and partial labeling: it cuts off part of the flow, and analyst confirmations become the training sample.
We estimate volume by the number of rare events, not rows: we fill the deficit with features from adjacent periods.
Does this require a separate loop?
No. We deploy in the existing infrastructure: the same queues, storage, and orchestration, role-based access, request log. We separate trigger computation from retraining by load so that review does not wait for recalculation.
How do we explain decisions and control the model after launch?
Each trigger comes with a set of features that influenced it: the analyst sees the cause, not an abstract score, and can challenge the result.
After launch, we monitor the distribution of input data and the share of confirmed triggers; model and dataset versions are recorded, and rollback is a standard operation.
Let's discuss the task: what to clarify before work starts
To come back with an estimate and plan rather than follow-up questions a week later, we need to remove basic uncertainties about the task.
Below is what determines the architecture of the AI risk manager and the implementation order: without these inputs, any estimate will be guesswork.
- Data sources and their state. Which systems provide transaction history, in what form, whether past incidents are labeled, and how long that has been maintained.
- Decision boundaries. Which risks we cover in the first phase — payments, counterparties, operational failures — and what we consciously leave outside the perimeter.
- Regulatory framework. Requirements for decision explainability, incident response times, and retention of scoring history.
- Decision point. Who receives the AI risk manager's signal — a human in the interface, an automatic rule, or a hybrid with confirmation.
- Target metrics. How we will know it has improved: share of missed incidents, false positives, time to escalation.
- Access and environment. Where we deploy — the client's cloud or a closed environment, and what model change approval mode is adopted.
- Who owns the solution after launch. Which team supports thresholds and rules, and whether knowledge and documentation transfer is needed.
Describe the task in free form — that is enough to start. We will come back with an estimate, a work plan, and a list of what will be required from your side at the start.
