Experts: Strike balance between cloud, on-premises AI

The playing field is leveling between cloud-based AI and on-premises AI in financial services, forcing institutions to strike a balance between the two. In 2025, 47% of financial institutions used a hybrid of cloud and internal systems for AI, up from 26% in 2024, according to Nvidia . Last year, 42% ran AI strictly on cloud architectures, compared with 57% in 2025. [caption id="attachment_57388250" align="aligncenter" width="800"] An Nvidia server displayed at the Nvidia GTC conference in San Jose, Calif. (Photo/Bloomberg)[/caption] Better prevention of “ shadow AI ” is one advantage of on-premises systems, Andrew Moore , chief executive of tech startup Lovelace and former head of Google Cloud AI , told FinAi News. Shadow AI refers to the use of an AI application without approval or oversight from a company’s IT department, according to IBM . For instance, someone who's used to using ChatGPT may open a workstation on a personal account for convenience, asking AI to “summarize this document or extract the values into a spreadsheet or something,” Moore said. “That data is then exfiltrating into some other organization’s systems,” Moore said. “It's not enough to just teach your employees to be sensible. You want to actually enforce it, and the way you enforce that is to make sure that the whole AI system is within its own tenant inside your bank, where you completely control.” Lovelace — an agentic AI provider for high-stakes decisions, including those in financial services — primarily builds through local models, training them on its isolated “context engine” while restricting access to the web or other outside entities, Moore said. Cost control David Moscatelli , CEO of on-premises AI provider Go.AI , similarly told FinAi News that “data never leaves your infrastructure” with local AI, giving FIs “unlimited usage, so you're not charged based on number of queries, tokens or agents.” Despite higher initial expenses compared to cloud, owning on-premises AI – often referred to as on-prem AI — “mitigates certain cost parameters” over the long term, he said. Purchasing on-prem AI typically costs between $79,000 and $335,000, with potential to break even in four months, according to agentic AI provider Arkeo AI . Deploying cloud-based AI at scale — 1 billion to 10 billion tokens per month — could result in monthly expenses ranging from $9,000 to $200,000. Hybrid approach Many major FIs are pouring resources into on-prem AI due to costs and data security. For example, JPMorgan recently partnered with AI infrastructure provider SambaNova , deploying hardware systems to power secure, on-prem AI for the bank, according to a July 8 SambaNova release. French bank BNP Paribas also expanded its partnership with LLM provider Mistral AI in May to strengthen on-prem solutions for sensitive processes such as KYC, BNP announced. [caption id="attachment_57388251" align="aligncenter" width="800"] (Photo/Bloomberg)[/caption] While on-prem is gaining favor, cloud AI still offers advantages including ease of deployment and third-party integrations, Moscatelli said. Cloud models are also more scalable and better suited for processing massive data sets, Tim Yalich , vice president of business development at Carleton , a tech solutions provider for lenders, told FinAi News . Ultimately, FIs must take a hybrid approach and focus on connectivity between systems while gradually modernizing their infrastructure, he said. “The market is looking for flexibility more than infrastructure labels,” he said. “Lenders don't wake up saying, ‘We need cloud.’ They wake up saying, ‘We need systems that connect.’ That’s really where AI comes into play.” Editor's note: This story has been updated to reflect Go Abacus' rebrand to Go.AI, effective Sept. 1. FinAi Lending Summit, set for Oct. 7-8 in Las Vegas, will include speakers from Fifth Third and Capital One as well as a fireside chat with Piermont Bank founder and Chief Executive Wendy Cai-Lee . To learn more about the 2026 event and register for early-bird pricing through Sept. 4, visit here .

Experts: Strike balance between cloud, on-premises AI

JPMorgan, BNP recently invested in on-prem AI solutions

5 questions with … BMO Head of Enterprise Fraud Management Spencer Shea

U.S. Bank’s Suzanne Rathbun featured fireside chat at FinAi Lending Summit

Related Posts

Why FICO built its own LLM

Employee feedback spurs AI innovation at Wintrust

Product roadmap, data governance key for AI vendor selection

U.S. Bank’s Suzanne Rathbun featured fireside chat at FinAi Lending Summit

EMERGING FINTECH DIRECTORY

FinAi Podcast

SPONSORED

The Next Generation of Cyberthreats: Powered by AI

Build an Antifragile Strategy to Outperform the Market

How AI and Product Experts Turn Fuzzy Requirements Into Focused Dev-ready Roadmaps

Connect

Welcome Back!

Retrieve your password

Continue Reading with FinAi News Premium - Less than $2/Day

Experts: Strike balance between cloud, on-premises AI

JPMorgan, BNP recently invested in on-prem AI solutions

5 questions with … BMO Head of Enterprise Fraud Management Spencer Shea

U.S. Bank’s Suzanne Rathbun featured fireside chat at FinAi Lending Summit

Related Posts

FinAi Podcast

Connect

Welcome Back!

Retrieve your password

Unlock This Article

Continue Reading with FinAi News Premium - Less than $2/Day