Update for minuet
This commit is contained in:
1 parent
226ac906a2
commit
f6921f3049
4 files changed
+30
-5
No files matched your search
@@ -7,7 +7,7 @@
|
||||
- [Reasoning effort per model](litellm-extra-body-reasoning.md) — GLM-5.3 defaults to reasoning_effort=max and cannot disable thinking; gpt-oss needs chat_template_kwargs via extra_body
|
||||
- [Reasoning vs completion tokens](reasoning-tokens-vs-completion-tokens.md) — reasoning comes out of max_tokens, and providers disagree about counting it
|
||||
- [Open WebUI image generation](owui-image-generation.md) — works via LiteLLM; the default 512x512 size is rejected by every image model
|
||||
- [Local code autocomplete](local-code-autocomplete.md) — FIM via /v1/completions; if it dies check rbw lock first, and never tune the coder model for decode
|
||||
- [Local code autocomplete](local-code-autocomplete.md) — FIM via /v1/completions; check rbw first; never tune for decode; ncmoe 22 + 8 s timeout since Viseron took VRAM
|
||||
- [NPU setting clamps memory](npu-setting-clamps-memory.md) — MSI NPU/AI BIOS options cost 17% DRAM bandwidth and the NPU is unusable by llama.cpp anyway.
|
||||
- [mainframe bad core](mainframe-bad-core.md) — APIC 20 (cpu10, an E-core) throws fatal MCEs; offlined by a unit, RMA candidate.
|
||||
- [crowdsec bans own devices](crowdsec-bans-own-devices.md) — "no external access" is usually crowdsec banning his own phone; http_path includes the query string
|
||||
|
||||
@@ -11,7 +11,10 @@ The LiteLLM stack serves two profiles with different permitted providers:
|
||||
**Anthropic is not an employer-approved vendor**, so `anthropic/*` is
|
||||
excluded there by policy, not by preference. PRC-origin models are not
|
||||
formally banned by his office but should be treated as prohibited until the
|
||||
security office gives written approval.
|
||||
security office gives written approval. That restriction is about HOSTED
|
||||
PRC-origin models (Moonshot/Kimi etc.). A PRC-origin model running locally
|
||||
(qwen3-coder-30b on llama-swap) is fine for work, since nothing leaves the
|
||||
machine. Connor's ruling, 2026-10-22.
|
||||
- **Personal** — `anthropic/*`, `openai/*`, and local.
|
||||
- `zen/*` is blocked for both, see [[zen-tos-hold]].
|
||||
|
||||
|
||||
@@ -69,3 +69,22 @@ identical (no ghost text), so check them in this order.
|
||||
Warm completions measure 0.98 s, matching the 0.7-1.3 s on record. A cold
|
||||
first request on a large file is still ~3.8 s and will exceed the 3 s timeout;
|
||||
raising `request_timeout` in `common/nvim/init.lua` is the open option.
|
||||
|
||||
---
|
||||
|
||||
**2026-10-22 — Viseron took VRAM; -ncmoe 20 -> 22; request_timeout 3 -> 8.**
|
||||
|
||||
Viseron (viseron + ffmpeg_camera_1) now holds ~666 MiB of VRAM, more than the
|
||||
449 MiB spare the tuned entry had, so qwen3-coder-30b failed to load at all
|
||||
("upstream command exited prematurely"). 21 still OOMs; 22 fits with ~440 MiB
|
||||
spare. Prefill fell to ~860 tok/s from ~1,100 (not A/B'd), so a fresh ~3k-token
|
||||
context took 4.8 s, past minuet's 3 s default. `request_timeout = 8` now set in
|
||||
`common/nvim/init.lua`. Warm, same-context requests are ~0.8 s.
|
||||
|
||||
A cold load is NOT lost when minuet times out: llama-swap finishes loading after
|
||||
the client disconnects (tested: ready ~5 s after a 3 s-aborted request). Cost is
|
||||
one missed completion. A BufEnter preload was offered and declined as too much;
|
||||
don't re-propose it.
|
||||
|
||||
llama-swap watches its config (poll, 2 s) — editing
|
||||
`~/config/llama-swap/config.yaml` needs no container restart.
|
||||
@@ -145,11 +145,11 @@ local function minuet_key()
|
||||
vim.fn.system({ "rbw", "unlocked" })
|
||||
if vim.v.shell_error == 0 then
|
||||
-- ⚠ The work laptop MUST use the work key: it is enforced
|
||||
-- server-side to 403 Anthropic and PRC-origin models, and a
|
||||
-- server-side to 403 Anthropic and hosted PRC-origin models, and a
|
||||
-- personal key here would silently remove that guarantee for
|
||||
-- everything nvim sends.
|
||||
local host = vim.uv.os_gethostname() or ""
|
||||
local entry = host:match("sctfw004") and "litellm-key-work"
|
||||
local entry = host:lower():match("sctfw004") and "litellm-key-work"
|
||||
or "litellm-key-personal"
|
||||
local out = vim.fn.system({ "rbw", "get", entry })
|
||||
if vim.v.shell_error == 0 and vim.trim(out) ~= "" then
|
||||
@@ -186,7 +186,10 @@ minuet.setup({
|
||||
-- Ghost text: there is no completion engine here (no nvim-cmp, no blink)
|
||||
-- and virtual text is the copilot-shaped one.
|
||||
virtualtext = {
|
||||
auto_trigger_ft = { "python", "rust", "lua", "c", "cpp", "julia", "sh" },
|
||||
-- Every filetype. The ignore list is UI buffers, where typing would
|
||||
-- send a request per keystroke into a picker.
|
||||
auto_trigger_ft = { "*" },
|
||||
auto_trigger_ignore_ft = { "TelescopePrompt", "oil" },
|
||||
keymap = {
|
||||
accept = "<A-a>",
|
||||
accept_line = "<A-l>",
|
||||
|
||||
Reference in new issue
Block a user