Home / Guides / LLM GPU break-even: find the volume where self-host wins
LLM GPU break-even: find the volume where self-host wins
Estimate the monthly token volume where rented or owned GPUs undercut hosted LLM API spend, and see how capacity and utilization move that point.
What break-even means here
LLM GPU break-even is the monthly token or request volume where estimated GPU monthly cost equals estimated API monthly cost for a chosen SKU and GPU setup. Below that volume, API usually wins. Above it, GPU can win if capacity still fits and quality is acceptable.
Break-even moves when catalog API rates, input/output mix, GPU $/hr, purchase amortization, hours, ops overhead, or tokens/sec change. API unit prices come from the catalog. GPU fields are editable reference defaults. Crossing break-even on dollars does not mean open weights match frontier API quality.
The simple break-even intuition
API monthly is roughly input and output tokens at catalog $/1M for the selected model. GPU monthly is roughly rental hours × $/hr (or purchase ÷ months) plus power and ops.
Break-even volume is roughly where those two monthlies meet. If you need two GPUs for capacity, GPU monthly rises and break-even moves higher.
Capacity can invalidate a paper break-even
A single GPU might look cheaper at average monthly tokens while failing peak tokens/sec or latency SLOs. Then you add GPUs, raise GPU/mo, and the break-even point slides.
Check required tokens per second at peak, sustained tokens/sec for your model size and stack, and headroom for retries. If the tool says capacity is tight, believe it.
How to run break-even in the calculator
Pick the API model you would replace from the catalog. Enter today’s monthly input/output tokens. Paste vendor GPU $/hr or purchase ÷ months as editable defaults. Set hours that match real uptime policy. Enter a sane tokens/sec from your bench. Add ops overhead.
Read break-even volume and capacity fit. Scale GPU count if tight, then re-read break-even. Sometimes switching API hosts beats self-hosting.
Common mistakes
Avoid these break-even mistakes.
- Publishing a break-even token count without stating GPU hours and tokens/sec
- Using 100% utilization in the break-even story
- Ignoring the second GPU required for peaks
- Treating GPU list prices as catalog truth
- Declaring victory without a quality gate
- Forgetting ops overhead until eng time appears
Frequently asked questions
- At what volume does self-hosting beat API pricing?
- At the break-even monthly volume where GPU/mo ≤ API/mo with capacity OK. Run it in /tools/gpu-llm-cost; there is no universal threshold.
- Is it cheaper to self-host an LLM or use an API?
- Depends on volume, utilization, GPU count, ops, and the API SKU. Break-even frames the cost half; quality is separate.
- How do I calculate GPU cost for LLM inference?
- Rental: $/hr × hours + power + ops. Purchase: price ÷ months + power + ops. Effective $/token needs useful tokens served.
- Can one GPU handle my monthly token volume?
- Only if sustained tokens/sec covers peaks with headroom. Monthly totals hide bursts. Use the capacity check.
- Is renting a cloud GPU cheaper than buying?
- Often yes before demand is stable. Purchase can win later at high utilization. Compare both modes with editable defaults.
- Does GPU utilization matter more than list $/hr?
- Frequently yes. Idle hours raise effective cost per served token and push break-even higher.
- Are break-even results using live verified GPU prices?
- No. GPU fields are editable reference defaults. API side uses TokenCalculator catalog rates. Verify GPU quotes externally.
- What’s a sane tokens/sec assumption for 70B?
- It depends on GPU class, quantization, context length, and batching. Bench your stack. Until then, use a conservative planning figure you are willing to defend.
- Self-host vs Groq Together break-even: should I include hosted inference APIs?
- Yes when those hosts meet your quality bar. Compare catalog API SKUs against GPU self-host rather than only against frontier prices.
- Our API bill is huge. Should we buy an H100?
- Check rental break-even and capacity first. Buying before utilization is proven is how idle hardware kills the savings story.
Try it in TokenCalculator
Enter catalog API volume assumptions and editable GPU defaults, then read break-even volume and capacity fit.
Open GPU / LLM cost calculator · Open cost calculator · Estimate API spend · All guides
Related tools
Related guides
GPU LLM cost calculator: compare API spend to self-host GPUs · Self-host vs API · When to self-host · GPU utilization
Sources and references
Official documentation used for definitions, counting methods, or rate cards. Always confirm critical budgets on the provider page.