dotfiles/common/claude/memory/-home-connor/reasoning-tokens-vs-completion-tokens.md

name, description, metadata

name	description	metadata
reasoning-tokens-vs-completion-tokens	Reasoning tokens come out of max_tokens, and providers disagree about whether they count as completion tokens	node_type type originSessionId modified memory reference 20b564c4-d665-445f-a1c7-25f9e6e666c1 2026-09-17T13:35:03.190Z

Two facts that together produce an empty HTTP 200 that looks like a model saying nothing:

1. Reasoning tokens come out of max_tokens. A cap set anywhere near the size of the expected answer is spent entirely on thinking, and the request returns no content. Already documented in ~/config/litellm/config.yaml's OpenAI block ("seen at max_tokens 16 and again at 500 on gpt-5.1-codex-max, which then worked unchanged at 4000").

2. Providers disagree about whether reasoning counts as completion. gpt-oss via llama.cpp folds reasoning INTO completion_tokens; glm/glm-5.3 reports completion_tokens: 0 beside completion_tokens_details.reasoning_tokens: 1023 for the same behaviour. Reading only completion_tokens shows a model that generated nothing.

finish_reason == "length" is the authoritative signal — check it before any token arithmetic. Add reasoning_tokens to completion_tokens whenever computing a rate or testing a budget; a decode figure divided by completion tokens alone reports a model that emitted 1,023 reasoning tokens and 12 of content as running at the rate of the 12.

Do not cap max_tokens to a budget in a benchmark. The cap silently becomes the thing being measured, and an empty 200 grades as "found nothing" — which on a code-review suite's defect-free control arm scores perfectly. Use a ceiling far above observed spend, record it in the result row, and treat an attempt that hits it as unusable rather than as an answer.

Related: litellm-extra-body-reasoning for turning the reasoning down.