connor
e6644d0616
Replaces the previous repo, which had split into two histories that never met (mainframe on a dead GitLab remote, the laptops on Gitea) with 146 dirty files across three machines and the NAS never enrolled at all. Branch-per-machine is gone. One main, with host differences expressed as small files under hosts/<hostname>/ rather than as branches, so there is nothing to merge. The reconciled zsh layer reduces 15-33 line forks to 1-7 effective lines per host; distro differences (oh-my-zsh prefix, syntax-highlighting path, fd vs fdfind) are probed in common/ instead. Fresh history: the old one carried six plaintext credentials, 45 MB of mail caches, browser caches and vendored binaries. 5,096 tracked files and 144 MB become 462 files and 2.6 MB. The .gitignore is now an allowlist, which is what keeps that true. Root cause of the rot: ~/.local/bin was a symlink to scripts/ with GOPATH inside it, so every go install wrote into version control (2.2 GB on the work laptop). PATH now points at the repo instead of the reverse. Also: Hyprland replaces sway and is sourced in two halves so $browser is defined before use; singleton automations carry ConditionHost= alongside host-layer-only placement; ddns moves from cron to a guarded timer; package manifests and pkg-snapshot/pkg-restore replace the X11-era install_scripts/; networkmanager-dmenu added to system76 (the binding always existed, the package never did).
4.4 KiB
Working notes for Claude on mainframe
LLM / inference stack
Connor runs a self-hosted inference stack: LiteLLM proxy in front of a local GPU
model (llama-swap), OpenAI, Anthropic, and Moonshot/Kimi, with OpenCode + OMO on
top. Before touching anything under ~/config/litellm/, ~/.config/opencode/,
or the model routing, read ~/guides/llm/robots.md. It carries the design
decisions, the measured data, and — more importantly — the specific wrong
conclusions this project has already reached and corrected. ~/guides/llm/README.md
is the short human version.
The five things most likely to waste an hour if you don't know them:
-
A config value is not applied until an invalid value produces an upstream 400. LiteLLM silently discards parameters it doesn't route (
store: falsebare inlitellm_paramsreturns 200 and never reaches OpenAI; it needsextra_body). -
Editing bind-mounted config does nothing without
sudo podman restart <container>. Changing an env var needs a fullsudo podman-compose up -d <svc>— a restart is not enough. The reverse trap is worse and bites silently:podman-compose up -dwill NOT restart a container when only a bind-mounted file changed, because the compose definition is unchanged. It prints the container name and does nothing. This applies to Grafana provisioning as much as tolitellm/config.yaml. -
If every hosted model hangs (not errors) while the host has internet, it's Docker:
dockerdsetsip filterFORWARD to policy drop and kills all podman egress. DNS still resolves, which makes it look like a broken vendor. Fix:sudo bash ~/config/host/podman-docker-forward.sh. Don't remove Docker — the Gitea Actions runners need it. -
HTTP 200 with empty content is not necessarily a bug. Three causes: an Anthropic refusal, a reasoning model eating its whole
max_tokensbudget before emitting text, or a dropped parameter. Local models do it too. Give reasoning models real headroom before concluding anything. -
Zen (
zen/*) is under a ToS block — make no calls to it, including "harmless" ones. Entries stay in config in an access group no key holds. -
The OpenCode sidebar under-reports cost — badly. It omits delegated child sessions, where most agent work happens: it showed ~$5.80 against a vendor-verified $17.39. Use
opencode statsor the LiteLLM spend log. -
Open WebUI gives every model its full built-in tool suite by default (
meta.builtinToolsdefaults to True when absent). Models that are weak at tool discipline loop onquery_knowledge_filesand friends even with no knowledge base configured. Symptom: many round trips with 25-30 token completions. Disable per-model in Admin → Models; there is no global switch. -
On this box, local inference is RAM-bandwidth-bound, not VRAM-bound. Decode speed is set by a model's active parameters, not its total, so an 80B MoE with 3B active runs at the same speed as a 30B with 3B active. Free VRAM buys prefill, not decode. Don't recommend a bigger GPU without checking which wall you're actually against.
-
The OMO agent aliases embed their model family on purpose —
personal/atlas-kimi-k2.7-code, notpersonal/atlas. OMO picks its per-model prompt by string-matching the last path segment of the model name, so a "tidy-up" rename silently disables every Kimi/GPT-specific prompt and guard with no error. Never shorten them.local/librarianandlocal/exploreare deliberately left un-suffixed —isGptModelmatches a bare"gpt"substring andgpt-oss-20bwould trip it.
Cost work: read model_group, never model, to attribute a request. Cost is
driven by cache-read volume and context size far more than by model choice.
General
- Repos here use an allowlist
.gitignore(**/*plus explicit!entries).~/config/and~/docs/are deliberately untracked — back up config edits by hand as<file>.pre-<thing>snapshots, which is the established convention. - Don't create project-level
CLAUDE.mdfiles in~— Connor keeps guidance at the~/.claudelevel and reference material in~/guides/. - Measure before concluding, and when a previous conclusion turns out to have been reached from bad data, record both the correction and why the original reasoning was unsound — otherwise someone re-opens it.