AI scaling squeezes software margins as token costs soar

Companies attempting to scale generative AI beyond the experimental phase are hitting an operational wall. The shift from isolated pilot programmes to widespread, autonomous workflows is triggering unforeseen financial and governance challenges that threaten to stall broader adoption.

The most immediate pressure is on profitability. According to Jannat Rajan, a growth investor at Forestay, companies that once enjoyed 80% to 90% gross margins are watching those figures drop to 50% or 60% when they embed AI into core operations. The culprit is unpredictable token costs—the computational expenses generated by large language models.

To survive this margin squeeze, businesses are adopting financial operations practices and repricing their software. Rajan notes that rather than routing all requests through a single model, organizations are building portfolios of lightweight, targeted AI engines. “This is very similar to the multi-cloud wave we had in the 2010s,” she says. “The principle is exactly the same: don't lock in entirely with one vendor but diversify and pick the right spend."

Financial constraints are not the only bottleneck. Thibault Martin, ecosystem lead at Dust, argues that technology rarely causes deployments to stall; rather, leadership misalignment does. Without a clear framework for security and budgets, executives operating under different incentives trap AI projects in endless loops. Martin suggests managing these new digital resources requires the same skills as managing human staff: defining success and providing clear feedback. “If the agent is producing something that's not useful, it’s often down to the humans who've been working with these agents and have not fully explained their expectations," he says.

Building trust in automated workflows

Building trust requires keeping humans in the loop. Lucien Bredin, cofounder of Naboo, says his company’s “AI Twin” manages 80% of event organization daily, leaving 20% to human account managers to maintain client confidence. At Box, solutions engineer Omar Davison emphasizes that early value often comes simply from saving employees 30 minutes a day before scaling to broader departmental goals. “Applying governance, training and change management are all ways we reduce the ‘principle of least surprise’ and ensure AI models’ outputs are reliable and correct,” he adds.

These shifting demands are rewriting recruitment and investment strategies. Naboo now ties 10% of employee performance evaluations to how they use AI agents, and Bredin advocates for hiring junior staff who are already accustomed to rapid technological change. For investors like Rajan, generic chatbots hold little appeal compared to founders with deep industry expertise and proprietary data. “When I meet somebody with genuine domain expertise who's collected data you haven't seen before and they build the intelligence on top of that — for me that's absolutely magic,” she says.