Three MoE quants, one 5090: what each one costs¶
The 5090 box now holds three mixture-of-experts models at once: Qwen3.8-Flash-Next on Basalt (my old production agent engine's model), GLM-5.3-Flash in Project Maya's Maya-M quant, and DeepSeek-V4.1-Flash in Maya's Maya-XS quant. All three are expert streamers - the model lives in host RAM, hot experts get promoted to VRAM, the rest streams per token - so they share the same physics and completely different numbers. I spent this week measuring them with the engines' own reported timings. The basalt-versus-strata story on the Qwen model already has its own post with the full A/B ledger; this one rounds out the picture with the two Maya quants, a context-length experiment that surprised me, and the quality data both quant makers publish.
The box and the ground rules¶
| GPU | RTX 5090 (32 GB) |
| CPU | Ryzen 7 9800X3D |
| RAM | 128 GB DDR5 |
| Storage | NVMe (DeepSeek's Engram tables read from here) |
| Engines | Basalt 0.3.2; Project Maya v1.1.65 (maya-glm, maya-ds4) |
One model runs at a time; the other containers are stopped, so every row below is a
solo measurement. All decode and prefill figures are the engine's own emitted lines
(Basalt's predicted_ms block, Maya's glm stat / ds4 serve lines), not client-side
math, and requests were issued engine-direct. Decode rates are measured over 256-token
windows, so they are short-window rates at the stated context.
One warning before the tables: all three engines use speculative decoding, and the draft acceptance rate is part of every decode number. Basalt's Qwen runs sat between roughly 50 and 80 percent acceptance; DeepSeek's 2-bit DSpark drafter kept 10 of 28 drafts (36 percent, 71 percent at the first position decaying to a third by the third). A decode figure from one of these engines is not directly comparable to another engine without its acceptance rate in hand.
Speed, measured¶
| model / quant | pass | context | prefill tok/s | decode tok/s | measurement type | Source |
|---|---|---|---|---|---|---|
| Qwen3.8-Flash-Next UD-IQ3_XXS, Basalt 0.3.2 | cold | ~1k | ≈1,340 (747 ms) | 174.6 | solo, engine-emitted | measured, Nov 11 ledger (post) |
| Qwen3.8-Flash-Next UD-IQ3_XXS, Basalt 0.3.2 | cold | ~64k | ≈11,800 (5,546 ms) | 268.2 | solo, engine-emitted | measured, Nov 11 ledger (post) |
| Qwen3.8-Flash-Next UD-IQ3_XXS, Basalt 0.3.2 | warm (cache hit) | ~64k | 8.4 ms resume | 267.2 | solo, engine-emitted | measured, Nov 11 ledger (post) |
| GLM-5.3-Flash Maya-M, maya-glm | short answer | <1k | — | 31.1-33.1 (median 32.2) | solo, engine-emitted, n=7 | measured, Nov 17 |
| GLM-5.3-Flash Maya-M, maya-glm | cold | ~4k | 1,868 | 24.0 | solo, engine-emitted | measured, Nov 17 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | short answer, cold box | <1k | — | 38.0-40.1 | solo, engine-emitted, n=4 | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | short answer, warm box | <1k | — | 50.3-51.0 | solo, engine-emitted | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | cold | 8k | 2,242 | 48.4 | solo, engine-emitted | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | cold | 64k | 4,581 | 51.0 | solo, engine-emitted | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | cold | 128k | 4,716 | 54.5 | solo, engine-emitted | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | cold | 207k | 4,480 | 56.7 | solo, engine-emitted | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | cold | 414k | 3,933 | 57.9 | solo, engine-emitted | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | cold | 829k | 3,081 | 53.9 | solo, engine-emitted, 271 s prefill | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | warm follow-up (full prefix reuse) | 128k | 5.0 s TTFT | 55.7 | solo, engine-emitted | measured, Nov 18 |
| DeepSeek-V4.1-Flash Maya-XS, maya-ds4 | warm follow-up (full prefix reuse) | 829k | 5.1 s TTFT | 55.5 | solo, engine-emitted | measured, Nov 18 |
Three things stand out to me:
The Qwen lane is a different league on speed. Basalt moves 268 t/s at 64k with 50-80 percent draft acceptance while DeepSeek at the same context does 51-55 with 36 percent. That gap is mostly the active-params class (Qwen's is a smaller brain per token) plus speculative decoding efficiency, not a Maya deficiency.
DeepSeek's decode improves with context. 38 t/s short on a cold box, 54-58 t/s at 207k-414k on a warm one. The long shared prefixes of a session warm the expert cache and the router profile, and the same happens with GLM (Maya's docs date-stamp this as expert-cache warmth, not a long-context bonus). Treat any single decode number from these engines as a band, not a point.
The cold-prefill scaling is a ramp, not a cliff. 4,581 t/s at 64k, 4,480 at 207k, 3,933 at 414k, 3,081 at 829k. That is the attention/indexer cost growing with prompt length showing up late and gently - an 829k-token first paste costs 271 seconds, and after that the prefix cache owns your session: the follow-up to that same 829k conversation came back in 5.1 seconds.
The context experiment: 256K vs 512K vs 1M cost nothing¶
DeepSeek-V4.1-Flash reaches 1M positions natively (nothing scaled), and Project Maya's ladder offers 32768 through 1048576. The setup recommends 262144 for this machine's memory, and I have previously taken on faith that a big context setting taxes the expert cache through the KV. So I ran the identical battery - four short decodes, salted cold prefills (a unique prefix per run so the prefix cache cannot help), and a deliberate-reuse follow-up - at 262144, 524288 and 1048576, engine restarted between each:
| task | 262K | 512K | 1M |
|---|---|---|---|
| 64k cold prefill (t/s) | 4,581 | 4,596 | 4,579 |
| 128k cold prefill (t/s) | 4,716 | 4,689 | 4,682 |
| 128k warm follow-up decode (t/s) | 55.7 | 55.0 | 53.0 |
Every column matches within noise. The setting itself is free: the context's caches (about 1.1 KB per position, FP4 KV plus index keys) are allocated lazily, so the box only pays for context it actually uses. VRAM held the same ~31.5 GB of expert cache at load across the runs (31.4 GB in the 262K engine logs, 31.7 GB measured at 1M) with 93 GB of RAM resident at 1M. The near-ceiling rows in the speed table above (207k vs 414k vs 829k) degrade with prompt length at every setting, confirming it is the prompt, not the setting. My box now runs 1M permanently, and the 2.5-minute restart between settings is the only cost the experiment produced.
Cross-family percentages don't compare
DeepSeek's 1M reach and GLM's are both native, but their KV designs are not equivalent (MLA latent caches versus GLM's streamed int8 KV), so "1M was free here" does not transfer as a promise about any other model or engine. It is a measured property of this quant, this engine, this box.
Quality: what the quant makers publish¶
Both Maya quants are made by the same person (peasantsmith) with GPTQ-style error-feedback rounding and importance statistics taken from the original model itself, and both projects publish token-level fidelity against the original on held-out text: how often the quant emits the same top token the full model would, and the next-token loss. That is the best public answer to "how much model am I actually running".
Against GLM-5.3-Flash (FP8 original):
| quant | size | same top token | zero-shot accuracy kept | Source |
|---|---|---|---|---|
| Maya-L | 156.3 GB | 90% | 99.2% | MAYA-M/L results, Project Maya repo |
| Maya-M | 116 GB | 86.2% (KL 0.329) | 97.9% | MAYA-M results, Project Maya repo |
| Maya-S | 96.5 GB | 83.3% | 97.9% | MAYA-S results, Project Maya repo |
| Maya-XS | 73.4 GB | no published numbers | — | — |
Against DeepSeek-V4.1-Flash (FP4/FP8 original):
| quant | model in memory | same top token | next-token loss | Source |
|---|---|---|---|---|
| Maya-S | 178.2 GB | 92.6% | +3.2% | model card, peasantsmith HF |
| Maya-XS | 115.3 GB | 87.5% | +8.7% | model card, peasantsmith HF |
Calibration points from the DeepSeek card worth knowing: the original weights run through the Maya engine score 96.4% same-token, so about four points of every quant's distance is the engine's own rounding, not the quantization. And the 161 GB n-gram Engram tables, which my box reads from SSD at Q6_K, measured "97.5% the same top token - the measurement's own noise floor", meaning they cost no measurable quality. Both quants keep every routed expert; nothing is pruned.
What these percentages can and cannot tell you
Same-top-token is measured against each model's own original, on that family's own held-out text. 87.5 (DeepSeek XS) against 86.2 (GLM M) is not a claim that one model is better than the other - different base model, different tokenizer. What it does say: my two live quants sit one tier apart from their originals' best published quant, and I pay roughly 2-4 points of token fidelity to fit a 300-750 billion parameter model into 128 GB of RAM. Nobody publishes a cross-family bake-off or a zero-shot table for the DeepSeek quants; that is a gap, not a guarantee.
Where each one earns its slot¶
The proxy fronts all of these and I route by workload. The lane assignment after a week with all three measured:
- Qwen3.8-Flash-Next on Basalt - voice, fast agents, anything where 250+ t/s and an 8 ms warm resume decide the experience. Fastest by roughly 5x.
- DeepSeek-V4.1-Flash Maya-XS - the default chat and agent brain right now. Biggest model I can hold, 1M context that costs nothing to enable, vision for free between pictures (the tower borrows ~319 MiB from prompt buffers while a picture is read and holds zero VRAM otherwise), and the Engram memory gives it an unusually good verbatim-recall streak.
- GLM-5.3-Flash Maya-M - tool-calling and front-end code. Its calibration text is deliberately weighted toward tool calls and HTML/CSS/JS work, it keeps 97.9 percent of the FP8 model's zero-shot accuracy, and on this box it is the slowest of the three (~32 t/s), which is why it is parked until its workload shows up.
For anything where being wrong is expensive, none of these percentages replace verification - quantization error compounds over long generations more than a 400-question multiple-choice table shows. But it is a good day when the honest answer to "how much model am I running?" is a published number instead of a guess.
Raw ledgers for the Nov 17-18 runs live on the box: bench-1165.json,
bench-ds4.json and bench-ctx-{262k,512k,1m}.json in the Maya working directory,
parsed from glm stat and ds4 serve engine lines.