“What 100 CFOs Reveal About the Real Cost of AI Compute” tackles one of the most urgent blind spots in modern finance: AI is no longer a “free add‑on” to the tech stack but a rapidly growing, often volatile, infrastructure cost that can materially reshape the P&L. In this Cloud Capital–GrowCFO session, Eric Keating (VP of Marketing, Cloud Capital) shares fresh survey data from 100+ CFOs and senior finance leaders at growth‑stage SaaS and tech‑forward companies, while Kevin Appleby (COO & Podcast Host, GrowCFO) and Colman Stephenson (CFO, Sprout.ai) ground the numbers in practical experience. Together, they show how AI and cloud compute are becoming the second‑largest line item for many businesses and why finance leaders can no longer treat these costs as a purely engineering concern.
The conversation highlights that AI infrastructure spend (including cloud, GPUs, data, and model usage) is already exerting significant downward pressure on margins, and that most organizations are struggling to forecast these costs reliably. AI‑driven workloads behave very differently from traditional SaaS: usage often scales linearly with cost, overruns frequently appear in core compute and storage rather than just token/API bills, and AI‑native businesses are already normalizing around gross margins in the 50–60% range instead of the traditional 70–85% SaaS benchmark. The session positions this shift as both a risk and a major opportunity: CFOs who build a deep partnership with engineering, understand the economic model of AI in their business, and implement structural governance can regain visibility, improve pricing decisions, and turn AI compute from an uncontrolled drain into a managed strategic investment.
Highlights:
- AI and cloud compute are becoming major P&L line items. Surveyed growth-stage technology companies report infrastructure spending at a median of 16–20% of revenue, with one-third spending more than 30%. This can make infrastructure the second-largest cost after payroll and, for some businesses, a potential future candidate for the largest.
- AI is compressing gross margins and widening the “AI margin gap.” AI-native product companies expect gross margins of around 52%, compared with the 70–85% traditionally associated with SaaS. Adding AI-related variable costs on top of existing COGS can reduce margins by 15–30 percentage points if those costs are not actively managed.
- Overruns are driven by core infrastructure, not just token bills. While many CFOs focus on model and API costs, the survey shows that unexpected overruns are most often found in compute and data storage. Ninety-six percent of companies reported at least one AI cost category exceeding forecast, while many are experiencing monthly cloud forecast variances of up to 10% or more.
- Classification and visibility are foundational. The discussion emphasizes the need to separate training, inference, and infrastructure costs, and then correctly allocate each between COGS and operating expenses based on production and non-production use. Poor tagging and allocation can cause companies to overstate COGS and understate margins.
- Joint finance and engineering ownership can outperform siloed approaches. Companies where finance and engineering jointly own cloud and AI cost management report stronger forecast accuracy, clearer driver-level visibility by product, customer, or team, and more effective cost governance than organizations where either function manages these costs alone.
- CFOs need to understand the economic model, not just the accounting. Modern finance leaders need to understand cost per interaction, cost per user, and margin impact; identify heavy-usage customer groups that may be cross-subsidized; and bring these insights into pricing, product, and investment decisions, particularly as AI becomes increasingly central to new products and business models.
Free AI Cost and Forecasting Playbook for Finance Playbook
Want to take the ideas from the session further?
Download the AI Cost and Forecasting Playbook for Finance for practical tools to help you assess how visible, predictable, and controlled AI spending is within your organization.
The playbook includes diagnostic questions, a cost-driver register, a forecast audit, and practical templates to help finance teams identify gaps, improve forecasting, and strengthen AI cost governance.
Download the playbook and use it to start a more informed conversation about AI costs across finance and engineering.
