Home / Guides / Why you must reserve output tokens

Why you must reserve output tokens

Input and completion share one token budget. If you fill the window with prompt text, there is nowhere left for the answer.

The shared budget people forget

A context window is not space for your document alone. It is space for everything the model must attend to in one request, including the tokens it will generate.

If the window is W tokens and you allow up to O completion tokens, the maximum input you can safely send is roughly W minus O (minus any extras your provider or SDK injects). Fill the window with input alone and the API may reject the call, truncate context, or fail when generation starts.

That is why TokenCalculator fit checks ask you to reserve output. The context window calculator exists to make the subtraction visible before production. Primer on windows: context windows. Overflow symptoms and fixes: context window overflow.

What “reserve” means in practice

Reserving output tokens means choosing the maximum completion length your product allows for that route, then treating those tokens as already spent when you plan input.

Window W is the upper bound from the catalog for your model SKU. Max output O is the longest answer format you allow; subtract it from W first. The input budget (system plus evidence plus history plus user) must stay under W minus O. Keep a safety buffer for provider overhead, growth, and logging wrappers.

O is usually your max_tokens or max completion setting (names vary by API). If the model stops early, you do not get those tokens back as extra input on that same request. The reservation is about the worst case you configured.

Failure mode: fits until the answer gets long

Teams often validate prompts with short completions in staging: a one sentence check, a tiny JSON object. Production asks for multi-section reports, long citations, or detailed tool plans. The same input suddenly overflows because O grew while input stayed large.

Other teams leave max_tokens very high just in case. That silently shrinks the safe input budget even when typical answers are short. You pay for that conservatism with more frequent context errors or forced truncation.

How many tokens to reserve for output (and why max_tokens shares the window)

Start from the product format. Short classifier labels need a small O. Structured JSON with fixed fields needs a medium O, sized to schema examples. Long narratives or multi-doc synthesis need a large O and a smaller input budget.

Tokenize sample outputs you consider acceptable. Do not guess from word counts alone. Use the tokenizer. Exact vs approximate guidance: exact vs approximate tokens. Tokenization background: tokenization.

Separate routes with different O values. A yes/no plus reason endpoint and a full brief endpoint should not share one oversized max completion. Different routes get different reservations and different input caps.

Leave margin. Providers may count special tokens, templates, or hidden wrappers. Keep a safety buffer beyond W minus O so you are not planning to the last token.

Worked planning example (no fake prices)

Suppose a model window is 128K tokens. Your product needs up to 4,000 completion tokens for a structured summary. Reserved output: 4,000. Safety margin: 2,000. Safe input budget: 128,000 minus 4,000 minus 2,000 equals 122,000 tokens for system, evidence, history, and user content combined.

If RAG evidence alone is planned at 100K tokens, history and system must fit in the remaining 22K. That is budgeting, not optimism. Full slice design: context window budgeting for RAG. Document-level workflow: context window calculator guide.

After fit is safe, estimate spend with the cost calculator. Output tokens often dominate cost even when they are a small share of the window. See prompt cost. Large prompts may also cross long-context pricing tiers. Use TokenCalculator catalog rates rather than invented unit prices.

Reservation vs quality vs advertised windows

Filling nearly all of W with input can still be a bad idea even when math says it fits. Attention quality can degrade on very long prompts. Advertised maxima are not always effective targets. Read effective vs advertised context window before you treat “W minus O equals maximum useful input” as a quality recommendation.

Implementation checklist

Define O per API route from real sample answers. Preflight: tokenize input; reject or reshape if input plus O exceeds W (with margin). Log input tokens, O, and utilization percentage. Alert on tight utilization before hard overflow. Re-run checks when system prompts, tools, or answer formats change. Confirm in the context window calculator during design reviews.

Common mistakes

Avoid these output reservation mistakes.

  • Filling the window with documents and leaving output to chance
  • Validating only with short staging completions
  • Setting a huge max completion for safety without shrinking input caps
  • Forgetting that JSON mode and long citations still consume output tokens
  • Using word heuristics instead of tokenizing sample answers
  • Ignoring safety margin for templates and special tokens
  • Fixing output overflow by silently truncating evidence users rely on

Frequently asked questions

Do output tokens count toward the context window?
Yes. Completion tokens compete for the same request budget as input. Reserve them when you plan fit.
Does max_tokens come out of the same window?
Yes. Your max completion setting is part of the shared budget. Treat O as already spent when you size input.
Do I need to subtract max_tokens from the context window myself?
For planning and preflight gates, yes. Many APIs will reject or truncate if input plus allowed completion exceeds W. Do not assume the provider will silently carve output space out of an already-full prompt.
How many tokens should I reserve for the reply?
Tokenize acceptable sample answers for that route and set O at or slightly above the worst case you allow. Short labels need a small O. Long JSON with citations needs more. Do not guess from word counts alone.
If the model stops early, can I send more input next time?
On the next request, yes, you can reshape the prompt. On the current request, the configured maximum still defines how much input was safe to send.
Is max output the same as average output?
No. Fit planning uses the maximum you allow. Cost forecasting should also consider typical output length using measured traffic.
What is the difference between finish_reason length and context overflow?
Hitting a length finish reason usually means generation stopped at your max completion while the request fit. Context overflow means the pack did not fit. Raising O without shrinking input can turn a length stop into overflow.
How does reservation interact with RAG?
Evidence budgets must be computed after subtracting O (and system, history, margin). See the context window budgeting for RAG guide.
What happens if I skip the reservation?
You risk overflow when answers get long: errors, truncation, or incomplete context. See the context window overflow guide.
Exact tokenizer or approximate for output samples?
Prefer exact (or best available) when setting hard caps. Approximate ratios are for early brainstorming only.
Where do I check this interactively?
Open /tools/context-window and follow the fit workflow in how to use a context window calculator.

Try it in TokenCalculator

Set your worst-case max completion, subtract it from the model window, then see how much input you may still send.

Open context window calculator · Open tokenizer · Estimate API spend · All guides

Related tools

Context window calculator · RAG cost · Cost calculator · Long-context pricing

Related guides

Context windows and overflow · Will my document fit? Context window calculator · Context budgeting for RAG · What happens when you exceed the context window · Effective vs advertised window

Sources and references

Official documentation used for definitions, counting methods, or rate cards. Always confirm critical budgets on the provider page.