Home / Guides
Guides
Practical explainers that always send you back to the same TokenCalculator calculator. 41 guides covering tokens, Exact vs Approx, pricing, caching, and provider doorways.
Main calculator · Tokenizer · Cost calculator · Compare · Tools
All guides
- What is a token in AI?A plain-language definition of AI tokens, why APIs bill in tokens instead of words, how counting works, and how to estimate cost before you call a model.
- What is tokenization?How LLMs split text into tokens with subword methods like BPE, why counts differ by model family, and how Exact vs Approx labeling works in TokenCalculator.
- Exact vs Approx token countsWhy TokenCalculator labels some models Exact and others Approx, how much heuristics can drift, and how to budget safely across OpenAI, Claude, Gemini, and other providers.
- Context windows and overflowWhat an LLM context window includes, why input plus output share one budget, how overflow fails, and how long-context pricing tiers change cost.
- How prompt cost is calculatedBreak down input vs output pricing, cache hits, batch discounts, monthly projections, and the workflow to estimate prompt cost before you ship.
- Prompt caching explainedHow prompt cache hits cut LLM input cost, when cache writes matter, which workloads benefit, and how to estimate savings in TokenCalculator before you ship.
- How LLM API pricing worksHow providers bill input vs output tokens, when cache, batch, and long-context tiers change the math, and how to estimate monthly spend before you ship.
- Tokens vs words: how to convertWhy LLM APIs bill in tokens instead of words, official English conversion rules of thumb, worked examples, and when to use Exact tokenization instead of a heuristic.
- OpenAI token calculatorCount Exact OpenAI tokens in your browser with o200k-style encodings, then estimate GPT API cost with cache, batch, and monthly volume.
- Claude token calculatorEstimate Anthropic Claude tokens and API cost in TokenCalculator with honest Approx labels, cache controls, and a clear path to Anthropic’s official count_tokens API.
- Gemini token calculatorEstimate Google Gemini tokens and API cost, including Flash vs Pro tradeoffs, Approx in-browser counts, and long-context pricing tiers when published.
- Will my document fit? Context window calculatorA practical workflow for checking whether a PDF, prompt pack, or chat history fits a model’s context window before you ship or spend.
- Context window budgeting for RAGHow to split a model’s context window into system, evidence, history, and output slices so retrieval apps stay reliable under real traffic.
- What happens when you exceed the context windowOverflow is a hard failure mode for chat, agents, and RAG. Learn the failure shapes, how to detect them early, and the fixes that actually restore reliability.
- Why you must reserve output tokensInput and completion share one token budget. If you fill the window with prompt text, there is nowhere left for the answer.
- Effective vs advertised context windowMarketing maxima tell you what can fit. Product quality often peaks earlier. Plan for an effective window, not only the advertised ceiling.
- Tokenizer comparison: GPT vs Claude vs GeminiPaste once, compare token counts across providers, and see Exact vs Approx labels so you know which numbers to trust for budgets.
- Why GPT, Claude, and Gemini token counts differWhy the same text produces different token counts on GPT, Claude, and Gemini: vocabularies, BPE vs SentencePiece, Exact vs Approx, and how to budget without drift.
- o200k vs cl100k: which OpenAI encoding counts your text?o200k_base versus cl100k_base: what changed, which GPT models use which encoding, and how to compare token counts without guessing.
- Stop using tiktoken for ClaudeWhy OpenAI tiktoken undercounts Claude, how Anthropic count_tokens gives Exact billing counts, and how to fix multi-provider cost estimates.
- How many tokens in a word? GPT, Claude, GeminiTokens per word for English, code, CJK, and emoji across GPT, Claude, and Gemini tokenizers, and how to measure efficiency without inventing prices.
- RAG cost calculator: estimate pipeline spendEstimate retrieval-augmented generation spend: corpus embedding, per-query retrieval packs, and LLM generation, then see which line dominates.
- RAG pipeline cost breakdownA stage-by-stage map of retrieval-augmented generation cost: embedding the corpus, retrieval, and generating answers, so you can see which line dominates.
- How to reduce RAG costsPractical levers to cut retrieval-augmented generation spend: retrieval shape, generation model, caching, output caps, and smarter re-embeds, checked against quality.
- RAG top-k retrieved context costHow top-k and chunk size set retrieved context tokens, drive generation input cost every query, and when raising k stops helping answers.
- Embedding vs generation cost in RAGWhy generation usually dwarfs embedding spend in retrieval-augmented generation, when index cost still matters, and how to compare the two lines with real rates.
- Embedding cost calculator: estimate corpus spendEstimate how much it costs to embed a knowledge base and keep query embeddings running: tokens × $/1M, batch vs standard, one-time index plus amortized refresh.
- OpenAI embedding pricing: text-embedding-3 planningHow OpenAI embedding API pricing works for planning: token × $/1M, text-embedding-3-small vs large tradeoffs, Batch vs standard, without inventing live unit prices.
- How to estimate embedding tokens for a corpusPractical methods to size tokens before you embed: tokenizer counts, docs × average, sampling, and overlap, so your embedding cost estimate matches the API bill.
- Batch embedding cost savingsHow Batch API discounts apply to embedding jobs, when standard rates still win, and how to model savings with editable rates instead of guessed live prices.
- Embedding reindex cost: how often to re-embedWhen to re-embed a corpus, how full vs incremental refresh changes spend, and how to amortize reindex cost beside query embeds without inventing live unit prices.
- Token visualizer: see colored token piecesFree token visualizer with Exact colored chips for OpenAI o200k_base and cl100k_base, plus counts and token IDs. No fake Claude or Gemini boundaries.
- How LLMs split text into tokensPlain-language walkthrough of subword tokenization, BPE-style merges, whitespace quirks, and why GPT, Claude, and Gemini disagree on the same sentence.
- tiktoken visualizer for o200k and cl100kOnline tiktoken-style visualizer for Exact OpenAI encodings: colored chips and token IDs for o200k_base versus cl100k_base on the same string.
- Why code and emoji use more tokensWhy JSON, source code, and emoji inflate token counts faster than English prose, and how Exact OpenAI chip maps make the density visible.
- Exact vs fake token boundariesWhy colored token chips are Exact only for OpenAI o200k_base and cl100k_base in TokenCalculator, and why Claude or Gemini piece maps in random UIs are often fake.
- GPU LLM cost calculator: compare API spend to self-host GPUsCompare monthly LLM API cost against self-hosted or rented GPU cost at your token volume, with a capacity check and a break-even estimate.
- Self-host vs API LLM cost: how to compare fairlyA fair cost comparison between calling a hosted LLM API and running open weights on a rented or owned GPU, including utilization, ops, and quality caveats.
- LLM GPU break-even: find the volume where self-host winsEstimate the monthly token volume where rented or owned GPUs undercut hosted LLM API spend, and see how capacity and utilization move that point.
- When to self-host an LLM (and when to stay on API)A practical decision guide for moving inference to your own or rented GPUs versus staying on hosted LLM APIs, based on volume, utilization, quality, and ops readiness.
- GPU utilization and LLM cost: why idle hours beat list $/hrHow GPU utilization, idle nights, and tokens/sec turn a cheap list $/hr into an expensive effective cost per token for self-hosted LLM inference.