The Proxmox balloon ate my model
A 125B MoE needs about 44 GiB of visible RAM to start. On the same boot, the VM went from 48.0 GiB to 15.9 GiB, and setup stopped with "smallest model needs 32 GB".
AI¶
Runaway thinking
Three requests hit exactly 65,536 completion tokens. 240, 252 and 287 seconds each. Thirteen minutes — 12% of a 1.75 hour window — in three requests, none of which delivered an answer.
Why two agents on one local LLM feel slow
I ran this for a while with two agents on the same engine and both of them felt sluggish, and I couldn't work out why. The engine reported good numbers. The experience didn't match.
The answer is the prefix cache, and it's structural — not a tuning knob.
Agent Configuration for Each Platform
In this post I'm going to discuss project and user agents for each of the platforms as well as native and third party applications that allow remote access to your agents.