dotfiles/common/claude/memory/-home-connor/litellm-extra-body-reasoning.md

name, description, metadata

name	description	metadata
litellm-extra-body-reasoning	How to set reasoning effort per model through LiteLLM, and which routes are measured vs assumed	node_type type originSessionId modified memory reference 20b564c4-d665-445f-a1c7-25f9e6e666c1 2026-09-17T13:42:14.834Z

GLM-5.3 (authoritative, from https://docs.z.ai/guides/capabilities/thinking):

• reasoning_effort is the control. Default is max — which means "think until done", not "think a lot". An unset GLM will spend an unbounded number of reasoning tokens on a large input, and that is documented behaviour, not a fault.
• GLM-5.3 / 5.3-Flash accept max, high, low ONLY. Not medium — an invalid value appears to fall back to the max default rather than erroring, which looks exactly like a hang.
• ⚠ GLM-5.3 cannot disable thinking. thinking: {type: "disabled"} is documented to error on 5.3/5.3-Flash. It is supported on 5.2 and earlier. Measured against the coding-plan endpoint it returned HTTP 200 and cut reasoning to 61 tokens anyway — an undocumented path with public reports of being both silently ignored and erroring (code 1210). Do not rely on it; use reasoning_effort.

gpt-oss-20b (local, llama.cpp): extra_body: {chat_template_kwargs: {reasoning_effort: "low"}} — 7,109 → 393 completion tokens, 148 s → 8 s on one identical prompt. Plain extra_body: {reasoning_effort: "low"} also works but cuts less deeply. Sent top-level it is a 400 UnsupportedParamsError naming "openai", which is LiteLLM's adapter for a custom api_base, not the vendor.

opencode (agent tier), settled 2026-09-18. opencode merges a model's options into the request body VERBATIM — no camelCase translation, no AI-SDK name mapping — so the value must be wrapped:

"options": { "extra_body": { "reasoning_effort": "low" } }   // works
"options": { "reasoning_effort": "low" }                     // silently inert

Proved binary with max_tokens: 1: flat and extra_body-wrapped both gave finish_reason: "length"; maxOutputTokens: 1 gave a normal reply. There is no model-level body field — opencode 1.18.25's schema has options, headers, limit, cost, reasoning, temperature, and no body. ⚠ Still unexplained: a flat reasoning_effort should have 400'd and returned 200 instead.

What is measured vs assumed. extra_body is a client-side mechanism (OpenAI SDK / LiteLLM) for putting arbitrary fields in the request body; z.ai itself documents these as ordinary top-level body fields, so the need for extra_body is a LiteLLM routing fact, not a vendor one. Confirmed by measurement for thinking on GLM (500 top-level), reasoning_effort on gpt-oss (400 top-level), and — tested 2026-09-18, streaming AND non-streaming — reasoning_effort top-level to GLM: 400, litellm.UnsupportedParamsError: zai does not support parameters. LiteLLM does NOT route it natively.

⚠ An extra_body 200 proves less than an ordinary 200. drop_params: false makes an unknown top-level parameter a loud 400, but an unrecognised key nested in extra_body is ignored and still returns 200 — so a knob can be set "successfully" and do nothing. Prove effect from token counts, never the status code.

Related: reasoning-tokens-vs-completion-tokens. Details: ~/guides/llm/robots.md §19, ~/docs/projects/llmbench/TRAPS.md.