Home / Guides / GPU LLM cost calculator: compare API spend to self-host GPUs
GPU LLM cost calculator: compare API spend to self-host GPUs
Compare monthly LLM API cost against self-hosted or rented GPU cost at your token volume, with a capacity check and a break-even estimate.
What a GPU LLM cost calculator actually estimates
A GPU LLM cost calculator answers one infra question: at this monthly token volume, is calling a hosted API cheaper than running inference on a GPU you rent or own, and can that GPU even keep up?
Most teams undercount both sides. On the API side they forget output tokens, system prompts, or retries. On the GPU side they price only list $/hr and ignore idle hours, electricity, and engineering time. Utilization and ops dominate more often than the sticker GPU rate.
TokenCalculator estimates monthly API cost from catalog rates, monthly GPU cost from editable rental or purchase defaults (plus hours, power, and ops), throughput capacity from tokens/sec versus required tokens, and break-even volume with a fit verdict. Do not treat GPU $/hr, purchase, tokens/sec, or power as catalog-verified. Verify with cloud vendors and electricity rates. Self-hosting open weights is not a drop-in for frontier API quality.
Cost components on each side
API bills input and output tokens at catalog $/1M and stays near zero when traffic is low. GPU cost accrues from rental hours or amortized purchase whether you serve one token or a million, plus editable power and ops overhead.
Quality risk differs too: the API side is the frontier or host SKU you chose; the GPU side is usually open weights that may lag frontier APIs. Compare cost separately from evals.
How to use the GPU / LLM cost calculator
Pick the API model you would actually call from the catalog. Enter monthly volume and input/output mix. Configure GPU rental $/hr or purchase ÷ months as editable reference defaults, plus hours and ops. Set a sane tokens/sec from your bench, not a heroic guess.
Read API/mo, GPU/mo, delta, break-even, and capacity OK/tight. If GPU/mo is lower but capacity is tight, you need more GPUs. If API/mo is lower at today’s volume, stay on API until volume or rates move.
Worked planning example (no invented unit prices)
Select the live API model from the catalog. Enter measured monthly input and output tokens. Paste your cloud vendor’s current GPU $/hr into the rental field. Set hours to match always-on vs scheduled run. Enter tokens/sec from your bench or a conservative planning default. Add ops overhead if eng time is material.
Compare monthly API vs monthly GPU, then read break-even and capacity. Change one lever at a time: fewer idle hours, a cheaper open-model API baseline, or a second GPU for peak.
Common mistakes
Avoid these GPU vs API cost mistakes.
- Treating GPU preset $/hr or purchase prices as verified catalog facts
- Comparing open weights on a GPU to frontier APIs on quality while only optimizing cost
- Assuming 100% GPU utilization when nights and weekends are idle
- Ignoring eng time until self-host looked cheap only on hardware
- Using peak tokens/sec from a short bench as sustained monthly throughput
- Skipping capacity: one GPU may not handle the monthly token volume at your SLO
- Buying hardware before testing a cloud rental break-even
Frequently asked questions
- What is a GPU LLM cost calculator?
- A tool that estimates monthly hosted API cost from catalog rates, estimates monthly GPU cost from editable rental or purchase defaults, and shows delta, break-even volume, and whether throughput can cover demand. TokenCalculator’s version is at /tools/gpu-llm-cost.
- Is it cheaper to self-host an LLM or use an API?
- It depends on volume, utilization, GPU hours, ops overhead, and which API SKU you compare against. Many teams stay on API until volume is high and traffic is steady.
- At what volume does self-hosting beat API pricing?
- At the break-even monthly token or request volume where GPU monthly cost undercuts API monthly cost, if capacity still fits. That point moves with rates, hours, and tokens/sec.
- How do I calculate GPU cost for LLM inference?
- Roughly: (rental $/hr × hours) + power + ops, or (purchase ÷ amortization months) + power + ops. Edit defaults; verify vendor and electricity rates.
- Can one GPU handle my monthly token volume?
- Only if tokens/sec × productive seconds covers required tokens with headroom for peaks and retries. Undersized capacity means more GPUs and a higher GPU/mo.
- Is renting a cloud GPU cheaper than buying?
- Often for uncertain or spiky load, because you avoid capex and can shut down. Buying can win at high, steady utilization after amortization. Model both modes with editable defaults.
- Are GPU prices in TokenCalculator catalog-verified?
- No. API rates come from the catalog for the selected model. GPU $/hr, purchase, tokens/sec, and power are editable reference defaults you must verify.
- Should I compare self-host cost to frontier or open-model APIs?
- Prefer an apples-to-apples quality bar. Open weights on a GPU vs a frontier API mixes cost with capability. If quality must match frontier, self-host may not be a substitute even when GPU math looks good.
- Our API bill is huge. Should we buy an H100?
- Maybe later. First measure volume, utilization, and whether a rental at your real hours beats the catalog API line. Buying locks capital before you know utilization.
- Idle GPU nights killed the savings story. Is that normal?
- Yes. Idle hours still bill on rental and still amortize on purchase. Utilization is often the real lever.
Try it in TokenCalculator
Pick a catalog API model, enter monthly volume, then set editable GPU rental or purchase defaults, hours, throughput, and ops overhead.
Open GPU / LLM cost calculator · Open cost calculator · Estimate API spend · All guides
Related tools
Related guides
Self-host vs API · GPU break-even · When to self-host · GPU utilization
Sources and references
Official documentation used for definitions, counting methods, or rate cards. Always confirm critical budgets on the provider page.