Home / Guides / RAG cost calculator: estimate pipeline spend

RAG cost calculator: estimate pipeline spend

Estimate retrieval-augmented generation spend: corpus embedding, per-query retrieval packs, and LLM generation, then see which line dominates.

What a RAG cost calculator actually estimates

A RAG cost calculator answers one operational question: for this corpus, this retrieval shape, this generation model, and this monthly volume, what do you pay, and which line dominates?

Most teams undercount because they price only the LLM answer. In production, every query usually sends system text, the user question, and a retrieved context pack billed as input tokens at generation rates. Embedding the knowledge base is a separate line. Query embedding is usually tiny. Vector database storage stays on your infra budget.

TokenCalculator’s RAG cost calculator estimates corpus embedding (with refresh), per-query generation from system + question + chunk tokens × top-k + output using catalog rates, optional query-embedding cost, and monthly totals. Pair it with the cost calculator, context window calculator, and cache savings tool. Do not invent unit prices: generation uses catalog rates; paste embed $/1M from your provider.

Cost components of a RAG system

Corpus embedding hits on index build and re-embed refresh. Query embedding hits each retrieval if you embed the question. Generation input hits every answer (system + question + retrieved chunks). Generation output hits every answer. Infra (vector DB, rerankers, hosting) sits outside the tool.

Generation input is where surprise bills come from. If each chunk is 800 tokens and top-k is 12, retrieved context alone is about 9,600 input tokens before system and question. Multiply by monthly queries and you see why RAG bills often look like “all LLM input from retrieved chunks.”

How to use the RAG cost calculator

Pick the generation model you will call from the catalog. Size the corpus in tokens and enter your embed $/1M and refresh factor. Shape the retrieval pack with average chunk tokens and top-k (retrieved tokens ≈ chunk × top-k). Add system, question, and realistic output. Set monthly query volume and optional cache hit rate for stable system prefixes.

Read which line dominates. If generation share is high, optimize retrieval shape and generation model first. If index share is high, you are re-embedding too often or the corpus is huge relative to traffic. Cap tokens, not only chunk count, and check fit in the context window calculator when packs get large.

Worked planning example (no invented unit prices)

Imagine a support bot with a mid-size handbook corpus, chunk size around 500 tokens, top-k of 8, a short system prompt, short questions, and 1,000 token answers at tens of thousands of queries per month.

Enter corpus tokens and your real embed $/1M. Set chunk tokens = 500 and top-k = 8 (about 4,000 retrieved tokens per query). Add system + question + 1,000 output. Pick the generation model from the catalog. Set monthly queries and any cache hit rate. Compare generation monthly vs index amortized vs query embed. Then change one lever at a time (for example top-k 8 → 4, or a cheaper generation SKU that still passes evals).

Common mistakes

Avoid these RAG cost-estimation mistakes.

  • Pricing only completion tokens and ignoring retrieved context as input
  • Using page counts instead of corpus tokens for embedding estimates
  • Raising top-k without measuring quality or input cost
  • Forgetting system prompt tokens on every call
  • Inventing embed or LLM unit prices instead of catalog / provider rates
  • Declaring the pack cheap when it overflows the context window
  • Re-embedding the full corpus daily when only a slice changed

Frequently asked questions

What is a RAG cost calculator?
A tool that estimates retrieval-augmented generation spend across indexing (embeddings), optional query embeddings, and LLM generation for a retrieval pack shape and monthly volume. TokenCalculator’s version is at /tools/rag-cost.
What are the cost components of a RAG system?
Corpus embedding (build and refresh), optional per-query question embedding, generation input (system + question + retrieved chunks), generation output, plus infra you track separately (vector DB, rerankers).
Why is generation more expensive than embedding in RAG?
Embedding is often priced at a low rate per million tokens and hits mainly at index time. Generation bills the retrieved pack as input on every query at LLM rates.
How many tokens of context does a typical RAG query use?
There is no universal typical. It is roughly system + question + (chunk tokens × top-k). Measure your production pack. The calculator makes that product explicit.
What does RAG cost per query?
Per-query cost is mostly generation for that pack plus a small query-embed line if you embed the question. Index cost is amortized across queries. Run your numbers in /tools/rag-cost.
How much does it cost to embed a knowledge base?
Corpus tokens ÷ 1,000,000 × your embed $/1M, times how often you fully re-embed. Paste the rate from your embed provider; do not invent it.
Are embedding rates from the TokenCalculator catalog?
No. Edit the embedding $/1M field from your embed provider. Generation uses published catalog rates for the selected LLM.
Does prompt caching help RAG?
Often yes when the system (and other stable prefix) is identical across calls. Cached prefix tokens can bill at a lower rate. Model and verify with /tools/cache-savings.
How do I reduce RAG costs?
Use a cheaper generation model when quality allows, lower top-k or chunk size, cache stable system text, cap answer length, and avoid needless full re-embeds.
My RAG bill is all LLM input from retrieved chunks. Is that normal?
Very common. Retrieved context is generation input. Shrink the pack or the model rate, then re-check quality.

Try it in TokenCalculator

Enter corpus size, chunk tokens, top-k, system and query tokens, output size, and monthly volume. Compare generation vs embedding share.

Open RAG cost calculator · Open cost calculator · Estimate API spend · All guides

Related tools

RAG cost calculator · Embedding cost · Context window · Generation cost

Related guides

RAG pipeline breakdown · Reduce RAG costs · Top-k and chunk cost · Embedding vs generation · Context for RAG

Sources and references

Official documentation used for definitions, counting methods, or rate cards. Always confirm critical budgets on the provider page.