nvim: local FIM autocomplete via minuet-ai, replacing the copilot that was off

copilot has been commented out on every machine with "deliberately off". This
is what replaces it: minuet-ai in virtual-text mode against qwen3-coder-30b on
mainframe's 3060. Nothing leaves the LAN -- llm.rcjohnstone.com has no public
A record, uses the internal CA, and Caddy source-IP gates it.

⚠ Route is /v1/completions, NOT /infill. Measured through the proxy:
/v1/completions returns 200 with FIM tokens in a raw prompt, /infill is 404
because LiteLLM does not expose it. llama.vim wants /infill, which would mean
talking to llama-server directly and bypassing the virtual key -- and on the
work laptop that key IS the policy boundary making Anthropic and PRC models
403. Not worth routing around for an editor plugin.

⚠ max_tokens is 32 on purpose. Measured on the real path at 48: `def add` came
back with `sub` and `mul` attached, so most of the latency was spent generating
text that would be discarded. Stop tokens include a blank line for the same
reason.

⚠ Expect occasional multi-second pauses: llama-swap is -np 1 with a ttl, so a
completion after another model was used pays a full reload. ~10 s cold,
0.7-1.3 s warm, measured end to end through the work key. Accepted because
concurrent agentic work is not viable on this hardware anyway.

Virtual text rather than a cmp source because there is no completion engine
installed here, and ghost text is the copilot-shaped one. Silent no-op when
LITELLM_KEY_NVIM is unset, so the headless NAS is unaffected.
This commit is contained in:
connor committed 2026-09-20 20:46:48 -04:00
1 parent 24dd2bf59f
commit 07149fe54a
2 files changed
+108

No files matched your search

+90
View File
@@ -64,6 +64,8 @@ vim.pack.add({
{ src = "https://github.com/nvim-treesitter/nvim-treesitter" },
-- copilot deliberately off on every machine
-- { src = "https://github.com/github/copilot.vim" },
-- ...and this is what replaced it: FIM completion off the local box.
{ src = "https://github.com/milanglacier/minuet-ai.nvim" },
})
require("mason").setup()
@@ -91,6 +93,94 @@ vim.lsp.config("rust_analyzer", {
}
})
-- ------------------------------------------------------------------------------
-- Local FIM autocomplete (minuet-ai -> LiteLLM -> llama-swap -> qwen3-coder-30b)
-- ------------------------------------------------------------------------------
--
-- Replaces copilot, which is off on every machine deliberately. Nothing leaves
-- the LAN: the model runs on mainframe's 3060 and `llm.rcjohnstone.com` has no
-- public A record -- internal CA, and Caddy source-IP gates it to 10.0.0.0/24,
-- 10.0.1.0/24 and the podman subnets.
--
-- ⚠ ROUTE: `/v1/completions`, NOT `/infill`. Measured 2026-09-18 through the
-- proxy: `/v1/completions` 200 with FIM tokens in a raw prompt, `/infill` 404
-- because LiteLLM does not expose it. `llama.vim` wants `/infill`, which would
-- mean talking to llama-server directly and bypassing the virtual key -- and on
-- the work laptop that key IS the policy boundary that makes Anthropic and PRC
-- models 403. Not worth routing around for an editor plugin.
--
-- ⚠ WHY qwen3-coder-30b: it is the best local FIM model by measurement (0.511
-- exact / 0.826 similarity against qwen3-next-80b's 0.302), and FIM is a
-- VOCABULARY feature -- gpt-oss-20b and gemma3-12b have no FIM tokens and
-- cannot do this at all. Its "skipped for capacity" verdict was about the 128k
-- agentic floor and does not bind here; autocomplete needs a few thousand
-- tokens of context, not 128k.
--
-- ⚠ EXPECT AN OCCASIONAL MULTI-SECOND PAUSE. llama-swap runs `-np 1` with a
-- ttl, so if another model was used in between, the next completion pays a
-- full reload. Measured: ~10 s cold, 0.7-1.3 s warm. Accepted deliberately --
-- concurrent agentic work is not viable on this hardware anyway.
local function minuet_key()
-- Set in ~/.secrets.zsh (untracked, mode 600) -- never in this repo.
-- The work laptop must point this at `litellm-key-work`, whose virtual key
-- is what enforces the no-Anthropic/no-PRC boundary server-side.
local k = os.getenv("LITELLM_KEY_NVIM")
if k and k ~= "" then return k end
return nil
end
if minuet_key() then
local base = os.getenv("LITELLM_BASE")
if not base or base == "" then base = "https://llm.rcjohnstone.com" end
require("minuet").setup({
provider = "openai_fim_compatible",
-- Ghost text, because there is no completion engine installed here
-- (no nvim-cmp, no blink) and virtual text is the copilot-shaped one.
virtualtext = {
auto_trigger_ft = { "python", "rust", "lua", "c", "cpp", "julia", "sh" },
keymap = {
accept = "<A-a>",
accept_line = "<A-l>",
prev = "<A-[>",
next = "<A-]>",
dismiss = "<A-e>",
},
},
provider_options = {
openai_fim_compatible = {
api_key = "LITELLM_KEY_NVIM",
name = "litellm",
end_point = base .. "/v1/completions",
model = "qwen3-coder-30b",
-- ⚠ KEEP THIS SMALL. Measured 2026-09-20 on the real path: at
-- max_tokens 48 the model happily continued past the useful
-- completion into whole extra functions -- `def add` came back
-- with `sub` and `mul` attached. Most of the latency was spent
-- generating text that would be thrown away. Raise it only if
-- completions are being visibly truncated mid-expression.
optional = {
max_tokens = 32,
top_p = 0.9,
stop = { "<|fim_pad|>", "<|endoftext|>", "\n\n" },
},
template = {
prompt = function(ctx)
return "<|fim_prefix|>" .. ctx.lines_before
.. "<|fim_suffix|>" .. ctx.lines_after
.. "<|fim_middle|>"
end,
suffix = false,
},
},
},
})
else
-- Silent by design: the NAS is headless and never needs this, and a
-- startup error on every nvim launch is how a warning stops being read.
vim.g.minuet_disabled_reason = "LITELLM_KEY_NVIM unset; see common/zsh/secrets.zsh.example"
end
vim.cmd("colorscheme vague")
-- KEYMAPS
+18
View File
@@ -10,3 +10,21 @@ export LASTFM_API_SECRET=""
# Read by the opencode profiles as {env:LITELLM_KEY}. opencode has no fallback
# syntax, so an unset value becomes an empty string and every request 401s.
export LITELLM_KEY=""
# ------------------------------------------------------------------------------
# nvim FIM autocomplete (minuet-ai -> LiteLLM -> qwen3-coder-30b)
# ------------------------------------------------------------------------------
# ⚠ WHICH KEY DEPENDS ON THE MACHINE, and on the work laptop it is the policy
# boundary, not a preference. `litellm-key-work` is enforced server-side to 403
# Anthropic and PRC-origin models; a personal key here would silently remove
# that guarantee for anything nvim sends.
#
# sctfw004 (work) rbw get litellm-key-work
# mainframe / system76 rbw get litellm-key-personal
#
# The NAS does not need it -- nvim disables the plugin silently when unset.
export LITELLM_KEY_NVIM="$(rbw get litellm-key-work 2>/dev/null)"
# Where the proxy lives. Mainframe talks to itself; every other machine uses
# the internal name, which has no public A record and is source-IP gated.
# export LITELLM_BASE="https://llm.rcjohnstone.com"