How Soaring AI Bills Are Vexing Wall Street Executives

(Bloomberg) -- No one ever wants to give a technology team a blank check. Yet for many chief financial officers, that's exactly what they were asked to do in the early rush to embrace AI. Now, after one-too-many surprise bills, finding a way to control costs has become an imperative.

It's a simmering issue in banks, where technology budgets are already under pressure from competing demands ranging from legacy IT modernization to cybersecurity. Some of the greatest hopes for productivity improvements are pinned on agentic AI, which can tackle complex tasks autonomously, but are inherently difficult to budget for as the amount of computing power each task consumes can be unpredictable.

"I actually don't think anybody in the organization has a good handle on it — if I am going to do XYZ, what is going to be my cost," said Bhavi Mehta, a partner at Bain & Co. and its global lead for advanced analytics in financial services. "As AI moves toward more sophisticated reasoning and agentic workflows, the tokens required for a given task can vary significantly. That makes understanding cost alongside business value increasingly important as banks scale AI," he added.

Only 12 of the 50 largest global banks tracked by AI benchmarking firm Evident have publicly attributed projected or realized returns to their adoption of AI, according to its latest report. Yet it's a rare CEO who doesn't talk up the potential benefits. Rising investment in the technology came up during third-quarter earnings calls at several US banks from Citigroup Inc. to Goldman Sachs Group Inc., but only a few executives had precise figures for their AI expenditures or the financial benefits they are generating.

So as the industry moves from the experimentation to the deployment stage, Bloomberg News spoke with more than a dozen engineers, executives and advisers about their experiences and strategies.

While tokenmaxxing — the practice of engineers using AI as much as possible — has become notorious — that's not always the cause of token usage blowout.

When Deutsche Bank AG's monitoring flagged individual employees consuming unusually large volumes of tokens, many turned out to be doing useful work, said Christoph Rabenseifner, the bank's chief strategy officer for technology, data and innovation. Instead of cutting them off, the bank steered that usage in a more productive direction. It also tested one task on a smaller, specially trained model instead of a large frontier model, cutting costs several times over, Rabenseifner added.