Agentic AI in financial services: what risk teams should know

It’s the end of Q1 and a new item sits on the model risk committee agenda: a deployment request for a system that drafts regulatory model documentation, interrogates data repositories, routes approvals between teams, and decides its own next step from what the last one returned. The requester calls it an AI assistant. The paperwork fills all the familiar fields: inputs, expected outputs, business rationale. From a distance it passes for a model submission. It isn’t one: nothing in the system estimates a risk parameter or scores a customer. It does work that people used to do. And the validation team has no framework for what it actually is.

A conventional model and an agentic system do not do the same job. The comparison that follows matters because the conventional model is what the validation toolkit was built around. A probability of default (PD) model maps a fixed set of inputs to an output through a function that doesn’t change between runs. An agentic system does a different kind of work entirely: it breaks a task into steps, calls external tools, and carries forward what it has learned so far, so its behaviour at any given step depends on what happened in prior steps.

That’s by design, not defect: the path-dependence is what makes the system useful. But it’s also what breaks conventional validation, which assumes a system will behave in production exactly as it behaved in testing.

The Bank of England and Financial Conduct Authority’s 2024 survey of 118 firms found 75% of respondents already using AI.¹ Foundation models, which include the large language models (LLMs) underlying most agentic systems, account for 17% of all reported use cases. Yet only 34% of those firms report a complete understanding of the AI they deploy. That gap isn’t a data quality issue; it’s a governance design challenge that is about to grow larger, and we unpack it in our companion piece on AI maturity assessment².

What are the parts of an agentic system?

The term "agent" is applied loosely enough to cover everything from a prompted language model to a fully autonomous, multi-step pipeline. Anthropic’s December 2024 practitioner guide draws the distinction the rest of this piece rests on.³ On its account a workflow is a system where an LLM and tools operate through predefined code paths, while an agentic system is one where the model directs its own process and tool use, deciding as it goes on the basis of intermediate results. That distinction maps closely onto the level of governance risk involved.

An agentic system has more parts than any one diagram shows, but three components determine most of how it behaves in practice, and they are the three that matter for governance.

1. Orchestrator: the language model that decides, at each step, which action to take and when the goal has been reached.
2. Tool layer: the system’s external capabilities: data retrieval, API calls, document search, code execution environments.
3. Memory layer: persists context across steps: what the model holds in its active working context, what it retrieves from an external store when it runs, and what is written to durable storage between sessions.

Of an agentic system’s three components, the memory layer carries the constraint that matters most for governance: the model’s working memory, its context window, is finite. When a run outgrows it, something has to give. Older steps are truncated, or compacted into a summary that keeps the gist and drops the detail. Either way the system can quietly lose access to its own earlier conclusions, and the validation team gets no signal when it happens.

Fixed inputs, moving behaviour

Conventional validation rests on two assumptions. First, the model you test is the model you run: its behaviour in production is exactly its behaviour in testing. Second, failure is observable: when performance degrades, the monitoring metrics show it. Backtesting is meaningful when both assumptions hold. Agentic systems violate both: their behaviour in production is not guaranteed to match how they behaved in testing, and their failures do not reliably surface in the monitoring metrics.

Three properties drive the divergence:

• Autonomous decision-making removes the step-by-step human review that provides a natural error check in rule-based processes
• Complex workflow management creates compounding failure paths that are difficult to reconstruct after the fact, because one system coordinates multiple downstream tools and sub-systems
• Open-ended inputs mean the system can be asked anything in natural language, so no test set built before deployment can cover everything it will face in production

The harder problem is that an agentic system can change behaviour with nothing to register it. A PD model degrades through concept drift: the world moves, the relationship between inputs and defaults shifts, and because real defaults eventually arrive, the metrics show the decline and the monitoring flags it. An agentic documentation tool has no equivalent signal. If the policy document it reads is revised, its output changes from that day on, with no realised outcome to compare against, no metric to degrade, and no model change to trigger revalidation. The document sits outside the model inventory, so nothing in the change process asks whether the system still does what it was approved to do.

Randomness makes it harder still. Language models deliberately introduce variation when they generate text,⁴ so two identical prompts can produce different reasoning chains and, in consequential cases, different outputs. That’s inherent to the architecture, and it undoes the output repeatability that makes conventional backtesting a meaningful safety check.

What existing frameworks can carry

No supervisory statement yet addresses agentic systems as such. The Prudential Regulation Authority (PRA) wrote its Supervisory Statement SS1/23, the UK’s model risk guidance, with statistical models in view. It says nothing about generative or agentic systems: the words appear nowhere in it.⁵ Our companion piece covers SS1/23 in full.⁶ Firms operating in the EU may fall inside the AI Act’s high-risk obligations, which draw no distinction for agentic architectures and phase in through 2026 and 2027, though which use cases are caught is still being worked out firm by firm. Three frameworks carry the weight in practice, and SS1/23 is the most applicable starting point.

SS1/23’s central principle is proportionality: the bigger the decision a model influences, the more scrutiny it gets. That tiering logic should extend to agentic systems. Agentic components that influence decisions falling within the model inventory’s scope can be pulled into the tiering framework using decision materiality as the criterion. For example, an agentic system that automates documentation drafting within an internal ratings-based (IRB) model change programme can be tiered alongside the models it supports rather than evaluated in isolation.

What still requires deliberate design is where the thresholds sit: how material a decision must be before an agentic system’s involvement escalates its tier. A workable starting point is inheritance: let the agent take the tier of the most material model it touches, and relax from there only with evidence.

Tiering determines how much scrutiny an agentic system receives. What that scrutiny should cover is a separate question, and it breaks into four domains:

• Safety: the system refuses out-of-scope instructions and fails gracefully when a tool returns something unexpected
• Reliability: output quality stays consistent as inputs vary
• Explainability: a human reviewer can reconstruct and audit the reasoning chain
• Alignment: behaviour stays within the intended boundary as the deployment environment changes

Safety, reliability and explainability map reasonably onto what SS1/23 already asks for: a conceptually sound choice of modelling technique and parameters under Principle 3, data quality management procedures, and ongoing performance monitoring shared between model users, owners and validators under Principle 4.⁵ Alignment is the gap, and not because SS1/23 ignores the environment: Principle 3.3 requires forward-looking testing against changes in economic and market conditions, and Principle 4.4 requires monitoring to judge whether such changes call for recalibration or redevelopment. Both assume a measurable performance signal that degrades. An agentic system’s behaviour can shift with the environment without any threshold being breached, because there is no stable output function to monitor.

Where are agentic systems already used in risk management?

Agentic systems aren’t a prospect for a future planning cycle. They are entering the risk function now, and two deployment areas are mature enough to carry practical lessons.

The first is IRB model development, where the burden is structural rather than a matter of effort. The Basel III finalisation package is Basel 3.1 in the UK and CRR3 in the EU, and still widely called Basel IV. It brings a revised output floor and internal model requirements that will force a wave of model redevelopment across the industry,⁷ and the constraint is people: on Gini’s reading of the pipeline, there are not enough experienced modellers to deliver that volume of change at the required pace by conventional means. That is why these systems are arriving regardless, and why the governance question is urgent rather than optional.

Commercial tooling has arrived: Oliver Wyman ships an agentic toolkit for credit risk model development, claiming a 65% cut in full-time-equivalent modelling costs and development cycles down from six months to two.⁸ Four tasks are already being handed to agents, and they are the same drafting, checking, and routing work the committee’s system at the top of this piece was built to do:

• Data quality review: agents that scan model inputs, flag gaps and anomalies, and produce structured quality summaries for modeller sign-off
• Variable selection: orchestrated loops that test candidate variables against statistical criteria and document results for governance review
• Documentation drafting: agents that draft methodology sections from structured model outputs, with human review before filing
• Approval routing: workflow orchestration that moves completed documentation packages to the correct reviewer tier based on materiality classification

The second is management stress testing. Much of the exercise is still slow manual work, and in a separate paper on that subject Oliver Wyman names five steps where agents are already being deployed: parameterising scenarios from previous exercises, monitoring geopolitical risk drivers, extracting structured data from documents, correcting quality problems in messy source data, and preparing decision material for stakeholders. The pattern in both areas is the same: the agent produces a structured first pass, a human reviews it before anything moves forward, and oversight sits where decision materiality demands it. That is SS1/23’s proportionality principle built in from the start.

What oversight does an agentic system need?

Everything to this point reduces to three problems. An agentic system’s reasoning can vanish from its own memory, so its behaviour can’t be reconstructed after the fact. Its value comes from autonomy, so no team can review its every step. And its inputs are open-ended, so no test set can cover what production will bring. The question for the risk team is therefore not whether the system counts as a model or an agent under the inventory definition, but whether the oversight infrastructure answers all three gaps.

Oversight infrastructure takes three elements, one for each gap.

The first is audit logging at every step: inputs, intermediate reasoning, tool calls, outputs. This answers the observability problem: the trace has to be kept as the system runs, because it can’t be recovered afterwards. Multi-agent architectures need additional handoff contracts between components, so each system checks what it receives before acting on it.

The second is human checkpoints tiered by materiality. No team can review every step an autonomous system takes, so the discipline is choosing where review matters: let routine steps run, and pause for a person wherever a decision could be material. SS1/23 already governs models this way, and the same proportionality carries across. What can’t be casual is the threshold itself: set it at approval, revisit it as the environment changes, and don’t leave it at default.

The third is adversarial testing alongside functional testing. Functional testing asks whether the system does its job on the inputs we expect it to see. With open-ended inputs, passing that test proves little. Adversarial testing asks the question that matters here: what happens on the inputs nobody anticipated? That means deliberately trying to make the system misbehave: instructions that push it outside its remit, tool responses in formats it doesn’t expect, context filled to the limit of the window. Functional testing shows what the system does most of the time. Adversarial testing shows what it does at the edge, which is where governance failures live.

Agentic systems are the sharp end of a broader shift: generative AI is settling into the industry’s core workflows, adoption is running ahead of understanding, and regulation is catching up from behind. None of that will pause while frameworks mature, and the committee that opened this piece will keep seeing requests like that one. The gap closes when the oversight layer matches what is actually there: an audit trail for reasoning that can’t be reconstructed afterwards, checkpoints wherever a decision turns material, and adversarial testing for the one thing SS1/23’s monitoring cannot catch without a performance signal, behaviour that shifts with the environment.

The upside is too large to leave on the table: systems that give risk teams back their hours and their judgement. Capturing it safely needs the regulation and guidance to mature alongside, so these systems are properly understood rather than merely trusted, and the firms that build the oversight now will be the ones free to move when they do.

Frequently asked questions