Home / Tools / GPU LLM cost
GPU / LLM cost calculator Compare catalog LLM API spend to self-host or cloud GPU cost at your volume. See break-even requests, capacity headroom, and which side wins this month.
Cost calculator · Cheapest API · RAG cost · GPU LLM cost guide
API side (TokenCalculator catalog) Comparison API model OpenAI. ChatGPT-4o Latest OpenAI. GPT-3.5 Turbo OpenAI. GPT-4 OpenAI. GPT-4.1 OpenAI. GPT-4.1 (2025-04-14) OpenAI. GPT-4.1 Mini OpenAI. GPT-4.1 Nano OpenAI. GPT-4 Turbo OpenAI. GPT-4o OpenAI. GPT-4o (2024-08-06) OpenAI. GPT-4o mini OpenAI. GPT-5 OpenAI. GPT-5 Mini OpenAI. GPT-5 Nano OpenAI. GPT-5.6 Sol OpenAI. GPT-5.6 Terra OpenAI. GPT-5.6 Luna OpenAI. o1 OpenAI. o1-mini OpenAI. o1-preview OpenAI. o3 OpenAI. o3-mini OpenAI. o4-mini Anthropic. Claude 2.1 Anthropic. Claude 3.5 Haiku Anthropic. Claude 3.5 Sonnet Anthropic. Claude 3 Haiku Anthropic. Claude 3 Opus Anthropic. Claude 3 Sonnet Anthropic. Claude Haiku 4.5 Anthropic. Claude Opus 4.6 Anthropic. Claude Opus 4.8 Anthropic. Claude Sonnet 4.6 Anthropic. Claude Sonnet 5 Google. Gemini 1.0 Pro Google. Gemini 1.5 Flash Google. Gemini 1.5 Flash-8B Google. Gemini 1.5 Pro Google. Gemini 2.0 Flash Google. Gemini 2.0 Flash-Lite Google. Gemini 2.0 Pro Exp Google. Gemini 2.5 Flash Google. Gemini 2.5 Flash-Lite Google. Gemini 2.5 Pro Google. Gemini 3.1 Flash-Lite Google. Gemini 3.1 Pro Google. Gemini 3 Flash Google. Gemma 3 27B DeepSeek. DeepSeek Chat DeepSeek. DeepSeek Coder DeepSeek. DeepSeek R1 DeepSeek. DeepSeek R1 Distill Qwen 32B DeepSeek. DeepSeek Reasoner DeepSeek. DeepSeek V3 DeepSeek. DeepSeek V4 Flash DeepSeek. DeepSeek V4 Pro DeepSeek. DeepSeek VL xAI. Grok 2 xAI. Grok 2 Mini xAI. Grok 2 Vision xAI. Grok 3 xAI. Grok 3 Mini xAI. Grok 4.6 xAI. Grok 4 xAI. Grok 4 Fast xAI. Grok Beta xAI. Grok Vision Beta Mistral. Codestral Mistral. Ministral 8B Mistral. Mistral Large (2407) Mistral. Mistral Large 3 Mistral. Mistral Medium 3 Mistral. Mistral Nemo Mistral. Mistral Saba Mistral. Mistral Small Mistral. Mistral Tiny Mistral. Open Mixtral 8x22B Mistral. Open Mixtral 8x7B Mistral. Pixtral 12B Groq. DeepSeek R1 Distill Llama 70B (Groq) Groq. Gemma 7B (Groq) Groq. Gemma 2 9B (Groq) Groq. Llama 3.1 8B Instant (Groq) Groq. Llama 3.2 11B Vision (Groq) Groq. Llama 3.2 1B (Groq) Groq. Llama 3.2 3B (Groq) Groq. Llama 3.3 70B (Groq) Groq. Llama 3.3 70B SpecDec (Groq) Groq. Llama 4 Scout (Groq) Groq. Llama Guard 3 8B (Groq) Groq. Mixtral 8x7B (Groq) Groq. QwQ 32B (Groq) Groq. Qwen3 32B (Groq) Cohere. Command A Cohere. Command Light Cohere. Command Nightly Cohere. Command R Cohere. Command R (08-2024) Cohere. Command R+ Cohere. Command R+ (08-2024) Cohere. Command R7B Together AI. DeepSeek R1 (Together) Together AI. DeepSeek V3 (Together) Together AI. Llama 3.1 70B (Together) Together AI. Llama 3.1 8B (Together) Together AI. Llama 3.2 3B (Together) Together AI. Llama 3.3 70B (Together) Together AI. Llama 4 Maverick (Together) Together AI. Mistral 7B (Together) Together AI. Mixtral 8x22B (Together) Together AI. Qwen2.5 72B (Together) Together AI. Qwen2.5 7B (Together) Together AI. Qwen2.5 Coder 32B (Together) Fireworks. DeepSeek R1 (Fireworks) Fireworks. DeepSeek V3 (Fireworks) Fireworks. Llama 3.1 70B (Fireworks) Fireworks. Llama 3.1 8B (Fireworks) Fireworks. Llama 3.2 3B (Fireworks) Fireworks. Llama 3.3 70B (Fireworks) Fireworks. Llama 4 Maverick (Fireworks) Fireworks. Mixtral 8x22B (Fireworks) Fireworks. MythoMax L2 13B (Fireworks) Fireworks. Qwen2.5 32B (Fireworks) Fireworks. Qwen2.5 72B (Fireworks) OpenAI · input $0.400/1M · output $1.60/1M · rates checked 2026-08-12
GPU side (editable reference defaults) GPU preset RTX 4090 class (consumer) L40S class A100 80GB class H100 80GB class Custom GPU Reference defaults. Spot and committed rates differ a lot.
Cloud rental Purchase amortize
Rental $/hour
API / request $0.0016
API / month $160.00
GPU stack / month $1482.96
Delta (API minus GPU) $-1322.96 (API cheaper)
Break-even requests / month 926,850
Break-even tokens / month 2,317,125,000
GPU capacity (tokens / month) 2,851,200,000
Capacity vs load 9% (OK headroom) How to use this Pick the API model you would pay today (or a fair open-model host comparison). Enter tokens per request and monthly request volume. Choose a GPU preset, edit $/hr or purchase terms, hours/day, tok/s, power, and ops. Read who wins this month, the break-even volume, and whether capacity holds. Open GPT-4.1 Mini in cost calculator · Self-host vs API · Break-even guide · When to self-host
Sources and references Official documentation used for definitions, counting methods, or rate cards. Always confirm critical budgets on the provider page.
FAQ
Is it cheaper to self-host an LLM or use an API? It depends on sustained volume, GPU utilization, ops cost, and which API you replace. Low volume usually favors APIs. High steady volume can favor a busy GPU. Run your numbers here.
At what volume does self-hosting beat API pricing? Break-even is roughly monthly GPU cost ÷ API cost per request (or per token mix). The calculator prints that volume for your inputs.
Are GPU $/hr and tokens/sec from the TokenCalculator catalog? No. Those are editable reference defaults. Always verify with your cloud vendor and a measured throughput on your model stack.
Should I compare a self-hosted Llama to Claude or GPT frontier APIs? Only for budget brainstorming. Quality, tools, and latency differ. Prefer comparing to open-model hosts (or the same open weights via API) when quality must match.