Banks inclined to build 'sovereign AI' infrastructure, IBM CEO says

Large banks are increasingly unwilling to run sensitive AI workloads on public frontier models and are looking to keep their data, logs and proprietary models inside their own walls. Sovereign AI — open-source models that run on local hardware — gained little traction in 2024, but is picking up steam in 2026, Chief Executive Arvind Krishna said during the company's third-quarter earnings call on Oct. 21. “Sovereign AI is definitely becoming a much larger conversation.” [caption id="attachment_57364639" align="alignright" width="431"] (Courtesy/Bloomberg)[/caption] A banking executive outside of the United States who is also an IBM customer told Krishna that he plans to test larger frontier models but wants to limit access to underlying data and logs to AI models, Krisha said during the call. A trader he spoke with took a similar stance, saying proprietary models won’t be available outside the firm. While large frontier models are not intentionally stealing a company’s data, they are training on it and sometimes the data gets left behind in conversation logs, Krishna said. “Sovereign [AI] becomes how do you make sure that none of that happens.” The need for sovereign AI in certain data-sensitive industries is going to turn into a tailwind for many players in the industry, “certainly the infrastructure and semiconductor players, but also for those of us who help enterprises deploy in a secure and in a private way,” Krishna said. IBM sees an opening on its mainframes with its Spyre AI accelerator as companies look to build sovereign AI infrastructure, he said, adding that IBM has nearly all the tools needed to help companies build and run their own on-premises AI. Gap is narrowing The performance gap between frontier and open models is shrinking, Krishna said. In late 2024, a broad benchmark might have scored a top frontier model at 70 out of 100 on accuracy and speed compared to an open model at 20 to 30, he said. By late 2025, the gap was closer to 70 out of 100 compared to 50 out of 100. Krishna expects the top model to reach 100 and the alternative 90. Open-weight models cost 10% to 25% of what frontier models charge per token, Krishna said, and an average large enterprise could get started with two or three racks of hardware. "It’s a simple question for any enterprise: Would you like something that’s 10% of the cost if it’s 90% as good as the best? Or do you want to pay that 10x higher cost?” Krishna said. The harder part is the tech expertise needed to configure these GPUs on enterprise data and build specific models. That gap likely persists for all but the most sophisticated enterprises, he said. Most tasks, such as summarizing emails, reviewing contracts or answering customer calls, don’t need a frontier model, he added. Some FIs including BNY and PNC are already building their own AI hardware and software, according to FinAi News' prior reporting. BY THE NUMBERS In Q3, IBM reported: Revenue of $17.1 billion, up 6% year over year; Infrastructure revenue of $3.7 billion, up 5% YoY; and Software revenue of $7.8 billion, up 9% YoY. Shares of IBM [NASDAQ: IBM] rose 0.26% from market open to $252 as of market close today. It has a market capitalization of $237 billion. Register here for early-bird pricing for the FinAi Banking Summit, taking place March 1-2 in Denver. View the full event agenda here .

Banks inclined to build ‘sovereign AI’ infrastructure, IBM CEO says

Agent identity becoming focus for enterprises

All employees become managers when using agentic AI

Related Posts

Mistral joins Nvidia’s open-model coalition

Ken Griffin says AI ‘almost certainly’ in a bubble right now

Fintech Ramp applies for trust bank charter with US regulator

EMERGING FINTECH DIRECTORY

FinAi Podcast

SPONSORED

The Next Generation of Cyberthreats: Powered by AI

Build an Antifragile Strategy to Outperform the Market

How AI and Product Experts Turn Fuzzy Requirements Into Focused Dev-ready Roadmaps

Connect

Welcome Back!

Retrieve your password

Continue Reading with FinAi News Premium - Less than $2/Day