‘Runaway agents’ pose threat to AI momentum

Agentic AI seems poised to revolutionize financial services, although there are risks that could stymie its momentum. The U.S. market for agentic AI in financial services is projected to reach $24.2 billion in 2034, up from $1 billion in 2025, according to research and consulting firm Market.us . However, “runaway agents” could disrupt the market, Kareem Saleh , chief executive of FairPlay , an AI enablement company for FIs and insurance providers, said last week at Fintech Meetup in Las Vegas. Runaway agents are AI agents that loop endlessly, execute unnecessary actions, bypass intended goals or retry tasks without learning, Demis Hassabis , CEO of AI research lab Google DeepMind , wrote in a Feb. 21 blog post. These errors often occur because of: Poorly defined goals; Missing logic that determines task completion; and Human-in-the-loop inadequacies. For FIs, the consequences of these errors could snowball quickly because AI agents travel in swarms, FairPlay’s Saleh said. For example, an insurance company may instruct an AI agent to respond quickly to customers, Saleh said. The ambiguity of this request could cause an agent to prematurely deny a claim, and a swarm of other agents may mimic this behavior shortly after. “Now you've got emergent misbehaviors like algorithmic collusion, where the agents tend toward anti-competitive behaviors,” Saleh said. These patterns also lead to “coordination failures,” which occur when agents misunderstand task boundaries or sequences, he said. “And then you've got this problem of cascading errors, where a mistake or a false assumption by one agent now becomes an input to a series of other agents,” he said. Other potential issues with runaway agents include data leaks and adversarial attacks that manipulate machine learning models, he said. Agentic AI risks are surfacing at a time when FIs are using the technology to drive efficiencies in areas including: Customer service ; Lending ; Sales ; and Payments . Register here for the FinAi Lending Summit, set for Oct. 7-8 in Las Vegas.

‘Runaway agents’ pose threat to AI momentum

Agentic AI risks caused by poorly defined goals, missing logic

90% of Citizens’ employees completed some form of gen AI training in 2025

European banks expect AI buildout to add headcount, survey says

Related Posts

Nvidia launches AI-driven fraud detection tool

Trump postpones signing AI security order over parts he disliked

National Bank of Canada taps Sardine for AI fraud prevention

European banks expect AI buildout to add headcount, survey says

Stay Informed with Our Newsletters

EMERGING FINTECH DIRECTORY

The Buzz Podcast

SPONSORED

How AI and Product Experts Turn Fuzzy Requirements Into Focused Dev-ready Roadmaps

Is Your Technology Supplier There for You?

Hiding in Plain Sight: How to Use Data to Spot Consumer Accounts Being Used by Small Businesses

Connect

Welcome Back!

Retrieve your password