Cut token bills without cutting staff

Companies facing rising AI token costs have reduced headcount to free budget for cloud spending. Some engineering teams have applied technical measures that sharply lowered token bills and offered a different way to manage costs.

At the close of GTC 2026, Nvidia CEO Jensen Huang warned on the All-In Podcast that an engineer whose annual AI token consumption fell below half of a $500,000 salary would be “deeply alarmed.” He also confirmed Nvidia is planning for about $2 billion in annual token spending for its engineering organization.

The four largest cloud providers have guided a combined roughly $700 billion in capital expenditure for 2026. Data from an outplacement firm shows AI was the most-cited reason for US job cuts for a fourth consecutive month. An internal memo at Meta recorded 8,000 role cuts in May during a quarter when revenue rose 33 percent.

A Gartner survey of 350 executives at firms with more than $1 billion in revenue deploying AI agents or automation found roughly 80 percent had cut headcount, with no clear link to improved returns. Analyst Helen Poitevin noted that reducing staff can create budget room without producing higher returns.

Uber provided AI coding tools to 5,000 engineers in December and exhausted its 2026 AI budget by April. The company’s chief operating officer, Andrew Macdonald, acknowledged that about 70 percent of committed code was generated by AI and said, “the link is not there yet.” Uber later imposed a $1,500 monthly cap per engineer after the overrun.

Engineering teams reported several practical levers to reduce token spending. Prompt caching stores static parts of requests so models do not reprocess the same text repeatedly; published price comparisons from major providers show repeated-input costs can fall by as much as 90 percent with caching. Security firm ProjectDiscovery restructured prompts and raised its cache hit rate from 7 percent to 84 percent, serving 9.8 billion tokens from cache and cutting total large-language-model spend by 59 to 70 percent.

Routing requests to smaller models for routine classification or summarization lowered per-token costs in many deployments. Provider price lists show flagship models can cost roughly five times more per token than smaller siblings. Batching non-real-time requests produced about a 50 percent cost advantage in some implementations. Retrieval-augmented generation reduces input size by sending only the relevant parts of a knowledge base to a model, and prompt compression removes redundant examples. Teams that run their own infrastructure shifted routine workloads to open-weight models at lower per-token cost.

Analysts and researchers described how companies redeploy savings. Poitevin’s analysis found firms that improved return on investment tended to use AI to amplify existing staff rather than replace them. Klarna replaced about 700 customer service roles with an OpenAI-powered assistant and later reported a decline in service quality; Chief Executive Sebastian Siemiatkowski said, “The result was lower quality, and that’s not sustainable.” Gartner projects that by 2027 half the companies that cut customer-service roles for AI will rehire those positions.

A Stanford institute study found employment for software developers aged 22 to 25 fell nearly 20 percent from 2024 levels while older cohorts grew, reflecting a reduction in junior-hire pipelines. Some analysts noted that engineering savings could create budget room to hire and train younger engineers if firms allocate funds that way.

Companies that combined tighter token controls with hiring and process changes reported improved financial outcomes in some cases. Firms have adopted spending caps, caching, model routing and batching as cost controls while some also maintained or restored roles tied to customer experience and long-term engineering capacity.