The Proxmox balloon ate my model¶
A 125B MoE needs about 44 GiB of visible RAM to start. On the same boot, the VM went from 48.0 GiB to 15.9 GiB, and setup stopped with "smallest model needs 32 GB".
Nothing crashed. No error in the hypervisor log. The engine just refused to start, and the reason it gave was a RAM gate.
Why the balloon does this¶
virtio_balloon is loaded inside the guest and the host can reclaim memory at runtime. The guest sees a MemTotal that shrinks. The engine measures /proc/meminfo MemTotal — not physical host RAM, not what the VM was configured with — so a live shrink changes what the engine believes is available.
That matters because the gate is measured, not configured:
| quant | needs visible |
|---|---|
| Q2_0 / IQ2_XS | ≥44 GiB |
| IQ3_XXS | ≥56 GiB |
| IQ3_S | ≥58 GiB |
The fix is host-side: pin the VM's memory so the host can't reclaim it. There's nothing to do inside the guest.
What I did about it¶
The swap script now preflights RAM and aborts before touching anything else. I'd not remove that guard — it's cheap and it prevents a swap that stops a healthy engine, fails on allocation, and leaves the GPU stranded.
Compare with history. A single reading tells you the current state; it doesn't tell you whether the state changed under you.
The knock-on effect¶
This is why quant upgrades are gated by RAM and not by VRAM. VRAM was fine — 31.8 of 32.6 GiB in use, which is by design, since the expert cache sizes itself to fill whatever the KV leaves. RAM is what blocks the upgrade.
And vision makes it worse. The encoder is a separate resident process holding a 907 MB model file, and container memory went from 35.76 GiB to 40.79 GiB — more than the encoder's own RSS, because of the warm-up buffers. Available RAM dropped to about 9 GiB.
So before assuming a quant upgrade is possible, check free -h. The balloon risk and the vision encoder both eat into the headroom, and a live MemTotal shrink makes the host-RAM cache get silently skipped. That's safe degradation — nothing breaks — but the cache stops helping and you lose the benefit without an error telling you so.