Agentic AI Raises the Stakes for Governance and Oversight
By Bassam Khattab, Partner, Advisory, Managed Services US Leader, KPMG
AI use cases in capital markets are abundant, but much of the industry conversation still focuses on capability: what models can do and how quickly they can be deployed. Getting ahead, however, will depend on addressing real, but less visible, constraints – those posed by risk, regulation, data sensitivity, and the need to maintain meaningful human oversight.
Those constraints are only intensifying as AI evolves. In April, the Office of the Comptroller of the Currency (OCC), along with the Federal Reserve and the Federal Deposit Insurance Corporation, issued updated risk management guidance, including a formal request for information on banks’ use of AI, across generative and agentic systems.[1] Implicit in that guidance is a clear expectation that firms must be able to monitor, explain, and intervene in how AI systems operate.
At the same time, more advanced agentic AI applications (systems that can take action, interact with other tools, and influence decisions across workflows) are redefining risks and the role of human oversight.
AI agents are now supporting investing and trading workflows, monitoring real-time market conditions, identifying opportunities, and executing predefined actions such as generating trade tickets or initiating trades. As applications continue to move into core operations like onboarding, KYC, and operational reviews, accountability requirements rise sharply. These challenges are amplified by the fact that most AI adoption in capital markets is layered onto decades-old legacy infrastructure, adding layers of technical complexity.
These changes redefine where risk sits in the AI lifecycle. Compliance expectations are changing faster than most firms can scale internally, particularly when it comes to AI model, cybersecurity, and third-party risk management. Meeting heightened demands requires governance to evolve from a checkpoint to a hardwired system.
Building trust by design
In capital markets, explainability is not optional. Regulators have made it clear that AI cannot operate as a “black box.” If a firm cannot explain how a model reached a decision, it cannot move that model into production. The priority is creating secure, controlled environments where AI can operate across workflows while preserving accountability, transparency, and human supervision at critical decision points.
Effective governance unifies oversight layers into a consistent framework for access, monitoring, and accountability. In practice, this means treating AI agents like privileged users: defining permissions, enforcing guardrails, and maintaining full auditability.
Leading organizations go further, embedding guardrails directly into agent behavior by defining how agents access data, constraining allowable actions, logging outputs, and establishing clear points for human intervention. They also build visibility into decision logic, performance over time, and how exceptions are identified and escalated.
As one AML executive at a Tier-1 bank explained, “it comes down to the type of data you’re putting into it. The more sensitive or customer-specific the data, the more likely the capability would need to be built or tightly controlled internally.”
But governance is more than a control framework. Agentic AI’s effectiveness depends on the strength of the surrounding ecosystem, with a strong data foundation, a defined operating model, and the right mix of human expertise alongside technology.
Execution, not ambition, is the constraint
The need for domain knowledge is becoming more urgent as firms are facing a skills velocity challenge, with capabilities evolving faster than internal teams can keep pace.
Continuously monitoring models, running internal change programs, managing AI risk, and keeping pace with evolving regulatory requirements places sustained pressure on internal teams. As automation absorbs repeatable work, firms need experts who can redesign workflows, data and engineering teams who can run AI reliably, and risk and security specialists who ensure models remain auditable and compliant.
The right AI implementation partners integrate accountability and human-in-the-loop across every layer of AI development and deployment, so systems are not only innovative, but responsible, resilient, and compliant.
The secret sauce for credible AI governance
Managed services are emerging as a practical way to operationalize AI governance at scale, delivering consistent execution, continuous monitoring, and clear escalation paths within the constraints of legacy environments.
With deep sector knowledge, they incorporate governance directly into workflows as AI scales. Continuous model monitoring and validation, combined with structured human-in-the-loop interventions, enable coordinated oversight across AI controls, cybersecurity, and operations.
As agentic AI becomes more embedded across capital markets, success will come from pairing human expertise with advanced analytics to improve decisions, manage risk effectively, and move faster. Achieving those outcomes requires more than technology. It requires strong data foundations, embedded governance, and clear accountability.
Modern managed services are defined by combining people, process, and technology. They move beyond traditional outsourcing by applying innovation, advanced tools, and a highly skilled workforce in day-to-day execution, allowing organizations to innovate with greater confidence, accelerate insights, improve execution, and drive efficiency while maintaining transparency and control.
Amid rising regulatory expectations and increasingly autonomous AI, credible governance is becoming a competitive differentiator. The firms that embed trust, accountability, and human judgment into their AI strategies will be best positioned to translate potential into sustainable business value.
[1] OCC Issues Updated Model Risk Management Guidance | OCC
