LLM providers race to scale AI infrastructure after tokens become profitable

The economics of AI has morphed and large language model providers are racing to expand the computing capacity that powers them. That dynamic came into focus on Nvidia 's first-quarter earnings call on May 20, when Chief Executive Jensen Huang argued that AI has moved from a speculative bet into a self-funding business. "Demand has gone parabolic. Agentic AI has arrived,” Huang said. “AI can now do productive and valuable work. Tokens are now profitable, so model makers are in a race to produce more.” [caption id="attachment_57380442" align="alignright" width="300"] (Courtesy/Bloomberg)[/caption] That idea that “tokens are now profitable” is central to why infrastructure spending is accelerating. ALSO READ: Big tech: AI CapEx spend has no brakes, supply chain disruptions remain a hurdle A token is the basic unit an AI model processing; tokens are the small chunks of chatbot inputs and outputs in the form of words and text, according to Nvidia. That means the cost of an AI workflow scales directly with how much work the AI model performs. Huang said the output now is monetizable. "In this new world of AI, compute is revenue,” he said. “Without compute, there's no way to generate tokens. Without tokens, there's no way to grow revenues.” The San Jose, Calif.-based company reported record quarterly revenue of $81.6 billion, an increase of 85% year over year. At the same time, per-token economics created a budget challenge for the institutions buying these AI tools. Rising token costs For banks and fintechs moving AI from pilots into production, token consumption is becoming a line item that needs active management, Sai Rangachari , chief product officer at core banking provider Temenos , told FinAi News . Temenos has "a good handle on token utilization, which can get pretty expensive," Rangachari said. "We have controls [on token usage] that we can put in place." The scale of potential spend is not theoretical. "There is a real token cost," Rangachari said, pointing to a widely circulated example of the creator of OpenClaw, who spent roughly $1 million on tokens in a single month. "Some people look at it as super productive use … but these costs can add up." Matching the model to the task to control usage is key, rather than defaulting to the most powerful — and most expensive — option, he said. Rangachari cited Anthropic' s Claude lineup, in which the top-tier Opus model is costly while Haiku and Sonnet are cheaper. "If you start doing model utilization in a smart way — like using Opus for planning, Sonnet for execution and Haiku for verification — then your cost comes down 70% to 75%," he said. Layering using open-source models , which carry only infrastructure costs, can push spending lower still. "It's not one size fits all." As providers race to scale on the conviction that tokens now pay for themselves, the cost discipline shifts downstream and depends less on any single model's price than on how intelligently institutions route work across them, Rangachari said. Register here for the FinAi Lending Summit, set for Oct. 7-8 in Las Vegas.

LLM providers race to scale AI infrastructure after tokens become profitable

Nvidia reported revenue of $81B, up 85% YoY in Q1

Trump postpones signing AI security order over parts he disliked

Plaid’s AI-driven LendScore can reduce lending risk by up to 41%

Related Posts

How U.S. Bank measures ROI of AI

Acquisitions bring AI consulting in-house

Vena grows context engine with Morpheo AI acquisition

Plaid’s AI-driven LendScore can reduce lending risk by up to 41%

EMERGING FINTECH DIRECTORY

FinAi Podcast

SPONSORED

The Next Generation of Cyberthreats: Powered by AI

Build an Antifragile Strategy to Outperform the Market

How AI and Product Experts Turn Fuzzy Requirements Into Focused Dev-ready Roadmaps

Connect

Welcome Back!

Retrieve your password

Continue Reading with FinAi News Premium - Less than $2/Day

LLM providers race to scale AI infrastructure after tokens become profitable

Nvidia reported revenue of $81B, up 85% YoY in Q1

Trump postpones signing AI security order over parts he disliked

Plaid’s AI-driven LendScore can reduce lending risk by up to 41%

Related Posts

FinAi Podcast

Connect

Welcome Back!

Retrieve your password

Unlock This Article

Continue Reading with FinAi News Premium - Less than $2/Day