Two engines, one model: Basalt versus Strata on the 5090¶
My production agent and voice traffic has run on Strata for months. A younger engine, Basalt, turned up with Qwen3.8-Flash-Next packs for the same class of hardware, so I spent a night putting them head to head on the 5090 box — first with different packs, then with the same pack, then with Strata's production quant converted into Basalt format so even the weights matched byte for byte. The short version: cold prefill is nearly the same story, and everything else is not.
The box and the ground rules¶
| GPU | RTX 5090 (32 GB) |
| CPU | Ryzen 7 9800X3D |
| RAM | 128 GB DDR5 |
| Engines | Strata 0.2.13 (driver 0.1.59), Basalt 0.3.2 |
| Model | Qwen3.8-Flash-Next, 512-expert hybrid MoE, 262k context |
One engine runs at a time — the other container is stopped, so every number is
a solo measurement, not a contended one. Both engines are expert-streaming
designs: the model lives in host RAM, hot experts get promoted to VRAM, and the
box streams the rest per token. Basalt's own startup log says the shape it
picked for the converted pack: ram-experts=37.6GB pool-affinity=auto pool=15.
Methodology¶
One script, both engines, identical requests: temperature 0, 256 output tokens,
single stream, engine-direct (no proxy hop). Three prompt sizes — ~800 tokens,
~12.5k, ~50k — each run cold (fresh conversation) and then warm: the same
conversation with a 7-token follow-up, which is exactly the shape agent traffic
generates turn after turn. Timings are the engine's own reported block
(prompt_n, cached_n, prefill_ms, decode). Raw JSONL ledger kept.
Decode numbers are measured over a short 256-token window at the given context, so treat them as short-decode rates. Draft acceptance sat between roughly 50 and 80 percent on every row of both engines — speculative decoding is doing heavy lifting in all of these figures.
Round 1: the unfair baseline¶
The first pass compared Strata running its production IQ3_S quant against Basalt running the official UD-IQ3_XXS pack. Different weights, so this is a sweep, not a verdict — Basalt swept every row (decode 20-45 percent higher, warm resume flat at 8.4 ms against Strata's 126-483 ms), but a weaker quant decodes faster, and that confound eats the whole result. Round 1 exists only to explain why I re-ran everything properly.
Round 2: the same pack on both engines¶
The clean test: the official Qwen3.8-FN-UD-IQ4_XS.basalt pack (97.6 GB) served
by both engines. Same weights, same quant, same machine, engines swapped.
| pass | engine | prefill ms | decode t/s | Source |
|---|---|---|---|---|
| 1k cold | Strata 0.2.13 | 752.6 | 98.4 | measured, engine-direct, solo |
| 1k cold | Basalt 0.3.2 | 747.3 | 174.6 | measured, engine-direct, solo |
| 16k cold | Strata 0.2.13 | 2161.3 | 144.6 | measured, engine-direct, solo |
| 16k cold | Basalt 0.3.2 | 1458.0 | 245.5 | measured, engine-direct, solo |
| 64k cold | Strata 0.2.13 | 7179.0 | 173.3 | measured, engine-direct, solo |
| 64k cold | Basalt 0.3.2 | 5546.0 | 268.2 | measured, engine-direct, solo |
| 1k warm | Strata 0.2.13 | 135.0 | 130.1 | measured, engine-direct, solo |
| 1k warm | Basalt 0.3.2 | 8.2 | 242.0 | measured, engine-direct, solo |
| 16k warm | Strata 0.2.13 | 209.0 | 162.2 | measured, engine-direct, solo |
| 16k warm | Basalt 0.3.2 | 8.3 | 256.5 | measured, engine-direct, solo |
| 64k warm | Strata 0.2.13 | 510.6 | 192.7 | measured, engine-direct, solo |
| 64k warm | Basalt 0.3.2 | 8.4 | 267.2 | measured, engine-direct, solo |
Three findings:
Cold prefill is close. At 1k they are within 1 percent of each other. At 16k and 64k Basalt runs 33 and 23 percent faster, which is a real difference but the same order of magnitude. Two young engines converging on the same expert-streaming prefill ceiling makes sense.
Warm resume is not close. Strata's warm pass costs 135 ms at 1k and grows to 511 ms at 64k — the restored prefix costs more the bigger it is. Basalt sits at 8.2-8.4 ms at every size, flat. For a chat curiosity that is invisible; for an agent making hundreds of cached turns a day it is the whole ballgame, and it's the number that made me pay attention.
Decode favors Basalt everywhere. +55 to +86 percent on the warm rows, +55 to +77 on the cold rows. Both engines show decode rising with context (Strata 98 to 193, Basalt 175 to 267, cold to warm-64k) — the drafter accepts more when the model has more context to lean on — so this is not a long-context penalty story, Basalt just moves tokens faster through the same architecture.
Round 3: converting Strata's production quant¶
The converter wants things it won't tell you about
Basalt serves its own pack format, not GGUF. The repo ships
tools/convert/convert.py, which takes --gguf plus --sections-from
pointing at an existing pack of the same architecture. It needs numpy — on
the engine box that means a dedicated venv — and it warns that S-series
llama.cpp codes (my production IQ3_S is one) can't be checked against a
reference without an imatrix file. The repo's precision docs state the
conversion carries existing codes across without requantising, so the
warning is about verification, not transformation.
I converted Strata's production Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S GGUF
(81.6 GB pack out, 252 seconds wall) and ran it on Basalt against the same
weights running natively on Strata:
| pass | engine | prefill ms | decode t/s | Source |
|---|---|---|---|---|
| 1k cold | Strata, native GGUF | 2265.2 (cold page cache, noise) | 179.1 | measured, engine-direct, solo |
| 1k cold | Basalt, converted | 635.5 | 248.2 | measured, engine-direct, solo |
| 16k cold | Strata, native GGUF | 1884.4 | 227.9 | measured, engine-direct, solo |
| 16k cold | Basalt, converted | 1345.3 | 320.9 | measured, engine-direct, solo |
| 64k cold | Strata, native GGUF | 6830.2 | 205.7 | measured, engine-direct, solo |
| 64k cold | Basalt, converted | 5298.8 | 296.4 | measured, engine-direct, solo |
| 64k warm | Strata, native GGUF | 483.1 | 227.5 | measured, engine-direct, solo |
| 64k warm | Basalt, converted | 8.3 | 297.2 | measured, engine-direct, solo |
The converted pack performs like the official packs — 248-343 t/s decode, 8 ms warm resume. Whatever the format boundary is, it is not costing performance, and the conversion path from an existing GGUF library takes four minutes per model.
What about quality?¶
I went looking for anyone who has published a quality delta between these two engines on shared weights and found nothing — no head-to-head evals, no perplexity reports. The honest position: same codes should mean the same model. The precision docs claim the conversion is bit-exact on the codes, and two different engines will diverge token-by-token anyway because kernel reduction orders differ and greedy near-ties flip. Statistically indistinguishable is the right expectation until someone publishes otherwise, but "no one has measured" is a gap, not a guarantee. The model card does carry the pack-author's own quality ladder across quants (IQ4_XS vs IQ3_XXS), so at minimum I did not change model quality between round 2 and round 3 by switching formats.
Where this leaves production¶
Strata still fronts llama.thomasjwilde.com. It is a version ahead of where
it was (0.2.13 shipped the DFlash2 drafter default and per-card tuning), it has
months of uptime under agent load, and its warm path is fast enough in
absolute terms. But an agent's life is warm turns — dozens an hour, all of them
follow-ups on a cached prefix — and Basalt's flat 8 ms resume against
Strata's context-scaling 135-511 ms is the one number here that would change
my daily experience if I moved. The candidate experiment is a split: keep
voice on Strata, point one agent at Basalt, and see whether the cache-turn
latency actually shows up in wall time over a week. Both boxes stay up, both
engines stay honest.