RESEARCH
Buying AI Infrastructure: Why the Lowest Rate can Cost More
A cheaper rate doesn’t always mean a cheaper bill.
Every AI project you start must eventually run in production. Whether it’s an internal assistant for employees, a customer support agent or a document-processing pipeline – all of these need a place to run, and there’s no shortage of pricing structures depending on where these applications live: for example - a monthly fee per seat, a price per million tokens, an hourly rate per GPU, or even a quote for a server.
It’s easy to compare these costs on a spreadsheet. But regardless of what the spreadsheet spits out, there’s a hidden story. A cheaper rate doesn’t always mean a cheaper bill.
AI usage is hard to forecast for several reasons: First, you don’t know how many tasks people will actually run - usage might stall or take off overnight for reasons you can’t predict. And second – you don’t really know how much compute each task will consume. An agent working through a complex, multi-step workflow can burn through magnitudes of tokens more than a simple chat response. When demand & usage are moving targets, your fixed spend often matters more than the variable rate.
Assuming you’ve shortlisted the options that meet your business, technical, or compliance requirements – the right choice is the one that makes financial sense even when your usage forecast is wrong.
Where AI runs today
Most organizations choose among four primary ways to run AI, and many use several at once:
- Buy software with AI embedded (e.g., Copilot)
- Pay a model provider per use – directly through a lab’s API (e.g. Anthropic or OpenAI API) or through a cloud provider’s (AWS Bedrock, Vertex AI, etc.) model platform
- Rent GPUs and run models yourself (from a major cloud provider like AWS or a neocloud like CoreWeave or Lambda)
- Buy and operate your own servers (e.g. NVIDIA DGX)
Each path has a different rate and its own considerations when usage drops:
Swipe horizontally to view the full comparison →
| Route | What you pay for | What happens to the cost when usage falls? | What you’re still responsible for |
|---|---|---|---|
| SaaS with AI embedded | Seats or subscriptions (sometimes with usage tiers) | Seat costs are usually locked in for the contract term | Setup, data access provisions, and driving adoption |
| Model provider API | Actual usage (input and output tokens) | Mostly shrinks, unless you agreed to a minimum spend commitment | Building the application, data retrieval, permissions, monitoring, and quality checks |
| Rented capacity | Computing time (on-demand or reserved) | Shrinks only if you wind down on-demand instances; reserved/committed capacity is fixed | All application-related work, plus hosting, running, and updating the models |
| Owned hardware | Equipment upfront, plus data center space, power and staff/support | Flat | Everything above, plus the physical facilities and hardware refresh cycles |
As a caveat - these are only general descriptions. The specific contract actually determines what your final bill looks like. For example, the model API path with a minimum spend component starts to behave more like a fixed cost in practice. Likewise, “renting” GPUs from a cloud provider doesn’t automatically mean just paying for what you use. Neoclouds like CoreWeave rely heavily on ‘take-or-pay’ contracts, where customers must contractually pay in full for a fixed amount of reserved GPU capacity over a set term - regardless of whether they use it or not.
Buyers also need to separate pricing discounts from guarantees on GPU capacity. For example, in AWS EC2, Savings Plans lower your hourly rate in exchange for committing to a set amount of hourly spending for 1-3 years, but capacity reservations actually hold the GPUs for you (and you pay for them whether you use them or not). A spending discount doesn’t by itself guarantee that the capacity you need will be available when your demand spikes
Evaluate based on business results & outcomes, not the inputs
As you’ve seen, vendors generally charge based on inputs (tokens processed, seats, GPU hours, etc.) versus business outcomes. Anthropic’s API, for example, bills for input and output tokens separately. These tell you what the model consumed, but not what it actually accomplished.
To compare more accurately, translate these charges from inputs into outcomes – specifically getting as close as you can to a “cost per result”: for example, a customer response accurate enough to send, a claim processed without errors, or a workflow completed on the first try. Every option you evaluate has to clear the same bar you set for accuracy and other factors.
An application running on a cheaper model or smaller GPU cluster that needs constant prompt retries & iterations or manual reviews can easily cost more per finished result than a higher-priced alternative (and any option that doesn’t clear your bar for other factors like latency, compliance, or security isn’t viable at any price anyway).
What happens when your demand forecast misses
To see how quickly different billing structures can flip the economics of an AI application, here’s a simple illustration:
Let’s say an organization is forecasting 10M accepted AI results or tasks per month. One vendor offers a purely usage-based rate of $0.12 per task. Another offers a discounted, flat commitment of $1M a month equivalent to $0.10 per task at the expected forecast.
Now look at what happens to the cost if adoption of the application takes longer than expected, and volume comes in at half the forecast:
Swipe horizontally to view both scenarios →
| Billing structure | At 5M tasks a month (50% of forecast) | At 10M tasks a month (100% of forecast) |
|---|---|---|
| Fixed commitment of $1M per month | $1M bill; $0.20 per task | $1M bill; $0.10 per task |
| Usage-based at $0.12 per task | $600K bill; $0.12 per task | $1.2M bill; $0.12 per task |
If the full forecast is met, the fixed commitment saves $200,000 per month. But at 50% of the forecast, the same commitment costs you $400,000 more per month, while also pushing your effective cost per task up to $0.20.
The same math applies to per-seat software when you buy licenses for the whole company, and only a fraction of employees use them regularly. Before signing a committed deal for a lower rate, calculate the break-even utilization threshold:
In this example, $1M ÷ ($0.12 × 10M) comes out to roughly 83%. This means that the fixed commitment saves money only if actual use stays above 83% of your forecast over the life of the contract.
The numbers here are illustrative, but before accepting a discount in exchange for a commitment, work out this number and ask yourself how confident you are of whether your demand forecast can clear it.
But demand can miss in the other direction too. Fixed capacity can only handle so much work, and if demand outpaces it, requests start to queue and responses slow down – unless you send the extra work to a pay-per-use service or rush to add capacity (which can take time when GPUs are scarce). Usage-based APIs scale more easily, although provider rate limits can still cap them.
Factor in the costs you won’t see on your quote
A quote covers only what the seller provides, not everything you’ll need to spend on. If you run models yourself on rented GPUs or your own servers, someone on your team still has to run the inference software, make patches, monitor uptime, and fix when it breaks. Dedicated staff/engineers, software licenses, and reserved capacity keep costing money even when the system is idle.
Owning your own hardware brings additional site costs too. Beyond the upfront cost for servers and other associated hardware, colocation providers like Equinix charge recurring, multi-year fees for power and rack space. If you pay for servers months before your colocation site has the power allocation to turn them on, you could be shelling out cash long before the hardware does any useful work.
To compare equally, evaluate your total “planning cost” over your target period:
Here, the “period capital charge” represents the cost of the hardware allocated to your planning period, using a consistent approach to depreciation or economic ownership cost. Account for financing costs or the opportunity cost of capital without counting them twice; expected resale value should reduce the net ownership cost rather than be added as an expense. Separately, track the upfront cash required and any contractual obligations that extend beyond the period you’re evaluating.
Guidelines to make a decision
No single path is right for every application, but some conditions consistently point in one direction versus the other.
Start with flexible, pay-per-use paths when:
- Demand is new, unpredictable, or spiky
- Total usage is low
- You don't have a dedicated team to run models
- Getting to production quickly matters most
Move towards commitments and self-operated paths when:
- Demand is high, steady, and already proven
- Strict data handling and residency requirements rule out 3rd party services
- You can spread fixed capacity across many workloads
- You already have the hardware and the people to run it
For most organizations, the safest path tends to be treating these paths as a sequence. Start on a flexible, pay-per-use service while you learn how people actually use your applications, and what each completed task truly costs in production. Once you have real data showing a steady baseline of demand, locking in reserved capacity or buying hardware for that proven volume can help lower your bill.