J.P. Morgan Payments is not sweating runaway AI token costs as the firm continues scaling AI usage internally and externally. 

As financial institutions grapple with rapidly escalating generative AI expenditures, Zack Anderson, chief data and analytics officer for global banking and payments at J.P. Morgan Payments, offered a notably calm assessment of the bank’s position.

“We tend to be pretty business-case driven,” Anderson said at J.P. Morgan Payments TechNYC on June 2. 

“We are looking at the benefits that we’re getting, which are quite good, and then trying to figure out how you manage the whole envelope.” 

The $3.6 trillion bank deploys AI where it makes sense rather than deploying it for the sake of deploying it, Anderson said. 

For example, the New York-based bank recognizes opportunities to expand operations and serve clients better by developing tools for agentic commerce and agentic treasuries, Anderson said.  

After the use case is well defined, he said, the bank commits AI fully to develop the tool. 

The cost 

In a March episode of the “All-In Podcast,” Nvidia Chief Executive Jensen Huang said that “if that $500,000 engineer did not consume at least $250,000 worth of tokens, I am going to be deeply alarmed.” 

“At the end of the year, I’m going to ask that $500K engineer, ‘How much did you spend in tokens?’ And if that person said $5,000, I will go [nuts],” he said. 

RBC, for one, reported during its earnings call that its token usage jumped by 500% year over year in its fiscal second quarter. 

Token costs vary depending on what model is used and how much it is used.  

Anthropic‘s flagship Claude Opus 4.7 costs $5 per 1 million input tokens and $25 per 1 million output tokens via API, meaning a heavy session generating just 100,000 output tokens costs $2.50, according to Anthropic’s website. 

For example, when Claude Opus 4.7 is given a three-page PDF to summarize, the model burns nearly 15,000 tokens, according to FinAi News’ analysis. 

Multiply that across hundreds of bankers, compliance officers and developers running multiple sessions daily and bills can soar quickly. 

After introducing Anthropic to its workforce, an unnamed company recently used $500 million in tokens in a month because it did not set usage limits, news publication Axios reported in May.  

Microsoft canceled its Claude subscriptions after costs skyrocketed with so many employees using it. 

Although Huang promotes liberal use of tokens, JPMorgan is taking a more conservative approach. 

“We didn’t go down the road of [having] AI token usage leaderboards or any of these weird things … as a result, we probably don’t have quite the problem,” Anderson said. 

Runaway token costs 

Mainstream publications such as Fortune and The Wall Street Journal have reported that AI tokens cost more than the employees they replace.  

FIs are being more cautious with how they deploy AI after feeling the pinch of “runaway AI token costs,” Sai Rangachari, chief product officer at banking services provider Temenos, told FinAi News. 

But he said cost is partly a model selection problem.  

“If you use Opus 4.7 for everything, you see costs here,” Rangachari said. “But if you start mixing and matching, the costs start coming down quite a bit.” 

Temenos uses state-of-the-art LLM models such as Opus 4.7 for planning and changes to less-powerful Sonnet 4.6 for execution, he said, adding that, Temenos and no other company can actually tell how many tokens they use in any given month. 

“It’s metric we dont track as of now,” he said. 

Payments company Stax also has a multi-LLM approach to AI, remaining aware of cost while deploying the tech, Mark Sundt, chief AI officer and former chief technology officer at Stax, previously told FinAi News. 

The company has a “pay-as-you-go” approach to AI costs, paying just for tokens it uses rather than paying for a licensing fee, he said. 

Another approach that FIs lean on is using more open-source and open-weighted models such as Mistral and Kimi, Rangachari added. 

“You can mix and match and manage the utilization costs,” Sundt said. “I think there are multiple levers emerging. I don’t think it’s a one-size-fits-all.” 

Register here for the FinAi Lending Summit, set for Oct. 7-8 in Las Vegas.