Big companies are rethinking 'out of control' AI spending — and turning to cheaper models

IBM ( IBM ) is calling attention to a growing trend among companies and their AI spending: They're keeping a closer eye on it.

"I think what we are seeing is that at least larger enterprises are becoming a lot more cognizant and a lot more aware of what the true ROI of all of their AI spend is," IBM CEO Arvind Krishna said during the tech company's earnings call on Wednesday.

"They are also getting worried if some of the AI spending is completely out of control with no real return," he added.

While hyperscalers have been ramping up capital spending on AI infrastructure and data centers, enterprise customers are increasingly turning to open-weight models , whose core components are public, as a more cost-effective way to implement AI.

"Our computation is that for an average large enterprise, you can get going with maybe two or three racks [of infrastructure]," said Krishna. "It gives you the ability to run those open-weight models currently, anywhere from 25% to 10% of the token cost of the frontier models."

Krishna noted that IBM's on-premises IT infrastructure is well-positioned for the shift, enabling the protection of intellectual property by running models locally or on private clouds.

Apollo chief economist Torsten Sløk recently argued that the key metric defining the AI build-out isn't physical capacity or power, but the price customers are willing to pay for the computing power being built. The costs of AI tokens have also fallen sharply.

(Disclosure: Yahoo is a portfolio company of funds managed by affiliates of Apollo Global Management.)

Companies are willing to pay a premium for computing power if it helps them increase revenue, improve profit margins, or cut costs. But those benefits have yet to materialize at a meaningful scale, Sløk has argued.

Frontier AI developers OpenAI ( OPAI.PVT ) and Anthropic ( ANTH.PVT ) are responding to enterprise demand for more affordable models.

OpenAI introduced lower-cost mid-range additions to its GPT-6 model family in September. Earlier this month, Anthropic unveiled Claude Haiku 5.5 , promising better performance at a lower cost than its predecessor.

Anthropic is on the march toward its highly anticipated initial public offering IPO w ith an S-1 filing potentially dropping as soon as Friday.

Ines Ferre is a senior business reporter for Yahoo Finance.