The playing field is leveling between cloud-based AI and on-premises AI in financial services, forcing institutions to strike a balance between the two.  

In 2025, 47% of financial institutions used a hybrid of cloud and internal systems for AI, up from 26% in 2024, according to Nvidia. Last year, 42% ran AI strictly on cloud architectures, compared with 57% in 2025. 

Better prevention of “shadow AI” is one advantage of on-premises systems, Andrew Moore, chief executive of tech startup Lovelace and former head of Google Cloud AI, told FinAi News.  

Shadow AI refers to the use of an AI application without approval or oversight from a company’s IT department, according to IBM.  

For instance, someone who’s used to using ChatGPT may open a workstation on a personal account for convenience, asking AI to “summarize this document or extract the values into a spreadsheet or something,” Moore said. 

“That data is then exfiltrating into some other organization’s systems,” Moore said. “It’s not enough to just teach your employees to be sensible. You want to actually enforce it, and the way you enforce that is to make sure that the whole AI system is within its own tenant inside your bank, where you completely control.” 

Lovelace — an agentic AI provider for high-stakes decisions, including those in financial services — primarily builds through local models, training them on its isolated “context engine” while restricting access to the web or other outside entities, Moore said.  

Cost control  

David Moscatelli, CEO of on-premises AI provider Go Abacus, similarly told FinAi News that “data never leaves your infrastructure” with local AI, giving FIs “unlimited usage, so you’re not charged based on number of queries, tokens or agents.” 

Despite higher initial expenses compared to cloud, owning on-premises AI – often referred to as on-prem AI — “mitigates certain cost parameters” over the long term, he said.  

Purchasing on-prem AI typically costs between $79,000 and $335,000, with potential to break even in four months, according to agentic AI provider Arkeo AI. Deploying cloud-based AI at scale — 1 billion to 10 billion tokens per month — could result in monthly expenses ranging from $9,000 to $200,000. 

Hybrid approach  

Many major FIs are pouring resources into on-prem AI due to costs and data security. 

For example, JPMorgan recently partnered with AI infrastructure provider SambaNova, deploying hardware systems to power secure, on-prem AI for the bank, according to a July 8 SambaNova release. 

French bank BNP Paribas also expanded its partnership with LLM provider Mistral AI in May to strengthen on-prem solutions for sensitive processes such as KYC, BNP announced. 

While on-prem is gaining favor, cloud AI still offers advantages including ease of deployment and third-party integrations, Moscatelli said.  

Cloud models are also more scalable and better suited for processing massive data sets, Tim Yalich, vice president of business development at Carleton, a tech solutions provider for lenders, told FinAi News.  

Ultimately, FIs must take a hybrid approach and focus on connectivity between systems while gradually modernizing their infrastructure, he said.  

“The market is looking for flexibility more than infrastructure labels,” he said. “Lenders don’t wake up saying, ‘We need cloud.’ They wake up saying, ‘We need systems that connect.’ That’s really where AI comes into play.” 

FinAi Lending Summit, set for Oct. 7-8 in Las Vegas, will include speakers from Fifth Third and Capital One as well as a fireside chat with Piermont Bank founder and Chief Executive Wendy Cai-Lee. To learn more about the 2026 event and register for early-bird pricing through Sept. 4, visit Event registration site.