What a Gen4 board costs: three engines, two R9700s, one benchmark ladder¶
Two Radeon AI PRO R9700s on a Z490 board with 62 GiB of DDR4 is not the machine the local-AI crowd writes about. The reference rigs in most engine docs are Gen5 AM5 boxes with fast DDR5. I wanted to know how much my Gen4 Rocket Lake board actually gives up, so I ran the same benchmark ladder against three engine/model combinations and put the results next to the published reference-rig numbers. Every figure below is engine-reported (not wall-clock math from me), and the raw JSONL sits in git- adjacent form on the box if anyone wants the per-request records.
The box¶
| Board | Gigabyte Z490 AORUS MASTER (i7-11700) |
| RAM | 62 GiB DDR4 (4x16, running at JEDEC speeds) |
| GPUs | 2x Radeon AI PRO R9700 TOP (32 GB each) |
| PCIe | Gen4; top card trains x16, second slot x8; no chipset peer-to-peer |
| Special | PCIe AtomicOps egress-blocking fix active — see the earlier post for why |
Kernel faults across every run below: zero. The atomic fix from the earlier post held through all of it, which is the only reason any of these numbers exist.
Methodology¶
One ladder, identical shape for every config: a ~69k-token cold prompt, an immediate
follow-up (1-3k new tokens on top of the cached conversation), then a ~140k-token
cold prompt and its follow-up. Filler text is salted per-size so "cold" means cold.
Timings come from the engine's own reported block (prefill rate, TTFT, decode rate).
For Strata I instead ran its native standard-v1 suite, because that is Strata's
own published measurement tool — comparing an engine's suite numbers to its own
docs is the only apples-to-apples that exists.
Radiance Flash-Next (MoE, TP2 across both cards)¶
Pack: qwen3.8-next-flash-fp8-iq4r-moe.rad (113.6 GiB, 4-bit codebook experts +
int8 trunk). Flags per the docs: --tp 2 --tp-wire wht6 --kv fp8 --prefix-cache-mib
4096 --host-pool-mib 12288 --num-speculative-tokens 3.
Fair rows are the C-262k-warm ones — the first 262k pass ran with a cold page
cache and read 70-95 tok/s decode, which is a storage warmup artifact, not a
model property. Warmth matters more than any context setting. The single
follow-up rows from that pass were themselves superseded by the G re-run
(see below) — follow-up decode needed n=4 to be trustworthy.
| tag | prompt_n | cache_n | prefill t/s | TTFT s | decode t/s | spec acc |
|---|---|---|---|---|---|---|
| C-262k-warm 64k cold | 69498 | 0 | 3552 | 19.6 | 117.0 | 42/72 |
| C-262k-warm 64k follow-up | 1940 | 67584 | 3542 | 0.6 | 141.7 (superseded) | 83/114 |
| C-262k-warm 128k cold | 140748 | 0 | 3452 | 40.8 | 138.2 | 45/57 |
| C-262k-warm 128k follow-up | 1510 | 139264 | 2635 | 0.6 | 101.4 (superseded) | 73/144 |
| G-rerun 64k cold | 65975 | 0 | 2816 | 23.4 | 65.8 | 79/120 |
| G-rerun 64k follow-ups ×4 | 465-572 | 65536 | 1420-1579 | 0.3-0.4 | 129.6 / 138.0 / 135.7 / 166.4 | 119-131 of 189-222 |
| G-rerun 128k cold | 133727 | 0 | 2879 | 46.5 | 86.1 | 82/120 |
| G-rerun 128k follow-ups ×4 | 633-778 | 133120 | 1485-1885 | 0.4-0.5 | 137.8 / 138.3 / 147.5 / 159.7 | 118-127 of 192-219 |
Cold prefill is flat with prompt size in both passes (C: 3452-3552, G: 2816-2879 at 64k/128k) — but the level itself moved ~20% between passes, same caution as the decode numbers. The prefix cache does its usual magic: follow-up TTFT drops from 20-47 s to under 0.6 s.
One claim from this ladder's first pass did not survive a retest. Reading single follow-up samples suggested decode sagging 15-25% at 128k. Re-running each context as a continuing conversation with four varied follow-ups (bullets, table, constrained prose, multi-step log entry; 192-token decode windows instead of 64-120) shows follow-up decode is steady and context-independent:
| follow-up decode (n=4 per context) | avg (min-max) | draft acceptance |
|---|---|---|
| 64k | 142 (130-166) | 54-69% |
| 128k | 146 (138-160) | 54-66% |
What is actually flaky is cold-run decode measured immediately after a huge prefill: the same config read 117-147 t/s one morning and 66-86 t/s the next, with draft acceptance nearly unchanged. Treat single-sample decode numbers from cold rows as noise; the follow-up rows were never the problem — they were just once adjacent to unusually fast cold rows.
Radiance 27B MXFP4 — one card, then two¶
Pack: qwen3.8-27b-mxfp4.rad (18.05 GiB, AMD Quark AWQ-MXFP4 weights — 4-bit
microscaling FP4, fp8 activations — with the DFlash2 drafter).
The model card claims this fits one 32 GB card at full context. It does, and the placement log says why: 18.07 GiB weights + 12.48 GiB KV. The KV is small because the architecture is hybrid — most layers are delta-net with fixed-size state, so long context is cheap.
TP1 (single card)¶
| tag | prompt_n | cache_n | prefill t/s | TTFT s | decode t/s |
|---|---|---|---|---|---|
| D-27b-t1 64k cold | 69164 | 0 | 2552.4 | 27.1 | 134.0 |
| D-27b-t1 follow-up | 3654 | 65536 | 1860.3 | 2.0 | 118.1 (superseded) |
| D-27b-t1 128k cold | 140102 | 0 | 1916.3 | 73.1 | 112.0 |
| D-27b-t1 follow-up | 2912 | 137216 | 1249.1 | 2.3 | 99.0 (superseded) |
The n=4 re-run (H-27b-t1, same protocol as the G rows) puts TP1 follow-up decode at
avg 116.0 (78.9-154.5) at 64k and 93.5 (62.7-110.7) at 128k; cold rows that pass:
126.9 / 109.8. The spread has a name: the constrained "exactly four sentences"
follow-up cratered draft acceptance to 21-27% in every pass, while the open-ended
follow-ups held 43-67% — content, not engine jitter.
A 27B dense model running on one card lands within about 15% of the Flash-Next MoE running on two cards. On this board, dense-plus-drafter simply doesn't pay the PCIe toll that MoE expert-streaming does.
The 263k probe¶
Requesting --max-model-len 262144 on a single card: the engine boots, but the KV
budget refuses to grow past 12.46 GiB. One card caps around 200k; no crash, no
refusal, just fit-to-available. Across two cards the picture flips — weights halve
to 9.68 GiB/card and KV expands to 20.04 GiB/card. Full 262144 fits with ~100 MiB
of headroom per card, and the ladder says nobody paid for it:
| tag | prompt_n | cache_n | prefill t/s | TTFT s | decode t/s |
|---|---|---|---|---|---|
| E-27b-t2 64k cold | 69164 | 0 | 3212.2 | 21.5 | 220.2 |
| E-27b-t2 follow-up | 3654 | 65536 | 2544.2 | 1.5 | 188.8 (superseded) |
| E-27b-t2 128k cold | 140102 | 0 | 2711.6 | 51.7 | 188.4 |
| E-27b-t2 follow-up | 2912 | 137216 | 1782.5 | 1.6 | 167.2 (superseded) |
TP2 splits the weights in half, so each card streams half the model per token on
its own memory pipe. The n=4 re-run (H-27b-t2) measures the real TP2 uplift:
follow-up decode avg 164.8 (113.2-189.5) at 64k and 158.5 (94.3-187.4) at
128k, cold rows 218.8 / 167.3. Against TP1's follow-up averages that is a
1.4x uplift at 64k and 1.7x at 128k — not the "nearly doubles" the
single-sample rows implied, though the cold-row comparison (127 to 219, 110 to
167) still runs 1.5-1.6x. Same content quirk as TP1: the constrained follow-up
craters acceptance (17-26%) and drags the range floor.
Strata IQ2_XS (MoE, layer-split across both cards)¶
Strata's own standard-v1 suite, profile launch-docker, IQ2_XS pack at its
(medians with min-max ranges across three full suite passes, each self-repeating
every case three times — n=9 per cell, concurrency 1; the single-pass numbers this
table ran with were optimistic by up to 17% on some cases — the same trap this post
keeps walking into)
65536 context ceiling, compose pinned with a hard mem_limit: 46g after the OOM
lessons of the same week. Medians of 3 reps, sanity phase 12/12:
| case | prompt_n | prefill t/s | TTFT s | decode t/s |
|---|---|---|---|---|
| short | 77 | 215 (160-233) | 0.38 | 88.4 (78.9-94.5) |
| 4k | 4093 | 1878 (1796-1948) | 2.20 | 73.5 (68.4-90.1) |
| 16k | 16387 | 2855 (2829-2878) | 5.77 | 73.0 (70.8-86.3) |
Strata's IQ2_XS is a smaller quant than the Radiance Flash-Next pack, and it's ~40-50% slower on decode. Placement, not bits: Radiance parks 21.3 GiB per card of expert codebooks VRAM-resident; Strata streams more expert mass over this board's Gen4 x8 link with the atomics blocked. The bigger quant wins because it touches the wire less.
Against the published AM5 numbers¶
Radiance's model cards quote their reference rig: MSI X670E Carbon, Ryzen 9 7950X3D, DDR5-6000 on XMP, 2x R9700 on Gen5 x8/x8 with peer-to-peer working. That makes their published figures a direct Gen5-vs-my-board comparison.
| Measurement | Published AM5 rig | This box (Gen4/DDR4) | Source |
|---|---|---|---|
| Flash-Next TP2 decode, 1 stream | ~160 t/s (15 ms/step, 2.4 accepted/step) | ~142-146 t/s follow-up avg (n=4, 64-128k; cold-run decode varies 66-147) | model card / measured 2026-11-08 |
| Flash-Next TP2 prefill | ~6400 t/s (26k prompt) | 3452-3552 t/s (C pass) and 2816-2879 t/s (G re-run) at 64-140k prompts; flat with prompt size within a pass | model card / measured 2026-11-08 |
| 27B MXFP4 TP1 decode | 122 t/s aggregate; 77 prose / 164 code (short prompts) | follow-up avg 116.0 (64k) / 93.5 (128k), n=4, range 63-155 | model card / measured 2026-11-08 |
| 27B MXFP4 TP2 decode | 204 t/s aggregate; 130 prose / 285 code (short prompts) | follow-up avg 164.8 (64k) / 158.5 (128k), n=4, range 94-190 | model card / measured 2026-11-08 |
| Strata IQ3_S decode, 1 vs 2 cards | 90-96 t/s on one R9700; 181-185 t/s on two (GSQ-RCO IQ3_S + MTP, matched 4K-input task, author-reported; that rig is AM5 but its cards sit at Gen4 x8/x4) | IQ3_S does not fit this box's 62 GiB at long context; IQ2_XS suite decode medians 73-88 t/s across prompt sizes (n=9, range 68-95), 139-142 t/s replicating his task shape (see below) | r/R9700 thread / measured 2026-11-08 |
The short-prompt aggregate figures and my ladder numbers aren't identical measurements — vendor cards use fixed 768-token outputs on short prompts, my ladder uses 1500-token outputs on long prompts — but the neighborhoods agree, which is itself the finding:
For the dense 27B, the Gen4 board costs essentially nothing. My TP2 numbers sit right at or above the reference rig's. VRAM-resident weights mean decode is GDDR6-limited, and both rigs have the same GDDR6.
For the MoE Flash-Next, the Gen4 board costs about 45% of prefill and about 11-27% of decode. The prefill gap maps almost exactly onto host RAM bandwidth: the reference rig's DDR5-6000 delivers ~96 GB/s, my DDR4 with four DIMMs manages ~47 GB/s, and cold-expert streaming during prefill is host-bandwidth-bound. The decode gap adds the second ingredient: their cards reach each other directly over Gen5 with peer-to-peer and working atomics; mine detour every tensor-parallel allreduce through host RAM because the Z490 chipset has no P2P — on top of the atomics workaround from the earlier post.
For context on the other end of my homelab, the same Strata IQ3_S model that won't even load here sustains 175-225 tok/s on the 5090 box with 128 GiB of DDR5 (measured in October). The quant was never the limiting factor. RAM bandwidth, PCIe generation, and peer-to-peer are.
Community anchor: Strata's second card¶
A r/R9700 build post carries the best community Strata number I've seen: a matched 1-vs-2-GPU A/B on the same 4K-input / 1,618-token-output JSON-copy task, Windows 11, 7800X3D, 128 GB DDR5 on an X670E-E, two R9700s at x8/x4, Strata HIP with GSQ-RCO IQ3_S and MTP. Author-reported, not independently verified — thread.
| Strata IQ3_S (4K task, matched A/B) | 1 GPU | 2 GPUs | Source |
|---|---|---|---|
| decode t/s | 90-96 | 181-185 | thread (author-reported) |
| prefill t/s | 619-649 | 1064-1068 | thread (author-reported) |
| process RAM | ~32 GB | ~11 GB | thread (author-reported) |
Two things make this worth the paper. First: the second card nearly doubles Strata decode, but for the opposite mechanical reason my Radiance TP2 doubled. Radiance splits the weights so each card streams half of them per token; Strata's second card lets it park more experts in VRAM instead of streaming them from host RAM — visible in that process-RAM column dropping from 32 GB to 11. Both engines, opposite levers, same conclusion: the second card pays for itself by touching the wire less.
Second: it independently confirms my box's problem was RAM, not the model. IQ3_S sustains 181-185 t/s on a 128 GB DDR5 machine even with the cards at x8/x4 — a worse PCIe shape than my board — and wouldn't complete loading on my 62 GiB.
Replicating his task on my box¶
Those 181-185 t/s can't be compared to my suite medians of 73-88 t/s — different tasks entirely. So I ran his task shape on my box the same evening: a JSON array in (4,247 tokens), verbatim copy back out (~1,700 tokens), temperature 0, decode measured excluding prefill, same as his method. My box ran its IQ2_XS pack; his ran GSQ-RCO IQ3_S with MTP — my engine log confirms Strata's drafting was on here too (93% draft acceptance, 1,290 of 1,384).
| Matched copy task | His rig | My box | Source |
|---|---|---|---|
| prefill t/s | 1,064-1,068 | 1,790 | thread (author-reported) / measured 2026-11-08, engine-reported |
| decode t/s | 181-185 | 139-142 | thread (author-reported) / measured 2026-11-08, engine-reported |
| cards | 2x R9700 at Gen5 x8 + Gen4 x4 | 2x R9700 at Gen4 x16 + Gen4 x8 | as stated by each |
| CPU / RAM | 7800X3D, 128 GB DDR5-5200 | i7-11700, 62 GiB DDR4-2933 | as stated by each |
| expert residency at decode | ~11 GB process RAM (streamed nothing) | 100% experts resident, KV 99.9% VRAM hits | thread / my engine log |
So on a like-for-like task, my Gen4/DDR4 box beat his prefill and trailed his decode by about 24% — while running the smaller quant and the better second-card link. What follows is explicitly speculation on my part, not the poster's claim and not something I measured: since neither the PCIe links nor the pack size explain the gap — both tilt my way — the most plausible remaining factor is the CPU platform. Strata's own source recruits the host CPU into the decode path (CPU prefill share, the draft layer computing overflow experts on CPU, thread pools sized by core topology), and a 96 MB 3D V-cache X3D is arguably the best desktop chip that menu can feed. But I know of no published Strata CPU-scaling ladder that would let me price that, and I won't invent one; treat "it's the CPU" as a hypothesis that survived elimination, not a measurement. The one hardware difference that did show its fingerprint in my data is RAM capacity: 128 GB is why his box can open IQ3_S at all.
What each setup costs your system RAM¶
Host RAM footprints measured per-container (docker stats cgroup), each engine
running alone, steady state shortly after healthy — not system-wide free(1),
which would drag in the OS and everything else on the box:
| Setup | VRAM placement (per card) | Host RAM required | Source |
|---|---|---|---|
| Radiance 27B MXFP4, TP1 | 18.07 GiB weights + 12.48 GiB KV, one card | 8.5 GiB (7.89 pinned) | measured 2026-11-08, docker stats + engine log (--placement all_vram) |
| Radiance 27B MXFP4, TP2 | 9.68 weights + 20.04 KV, both cards, full 262k | 8.5 GiB (same 7.89 pinned — TP collectives stage through pinned host memory) | measured 2026-11-08 |
| Radiance Flash-Next fp8-iq4r, TP2 | ~21.3 weights + KV per card | 38.6 GiB (37.1 pinned, incl. an 11.97 GiB/rank expert pool that streams over PCIe by design — the 113.6 GiB pack cannot fit 64 GiB of VRAM) | measured 2026-11-08, docker stats + engine log |
| Strata IQ2_XS, layer-split across both cards | expert cache ~18.2 GiB/card, KV on cards | ~40 GiB steady (40.16 GiB; my fence is 46 GiB — the planner/warmup peak wants more than steady and I have not measured its true unfenced ceiling) | measured 2026-11-08, docker stats |
Two practical reads: the dense 27B is the only one of these that coexists
comfortably on a 62 GiB box alongside a desktop and VMs — it is fully VRAM-resident
and its host side is scaffolding, not model state. The two MoE setups each want
~40 GiB of host RAM before anything else on the machine, and Strata's larger
IQ3_S tier will OOM a 64 GiB machine outright during load (several times
measured, always painfully). Add ~2-4 GiB OS baseline to any of these when sizing
a box, and expect running two MoE engines at once to roughly double the host-RAM
bill — Strata's --shared-expert-arena can share one arena Strata-to-Strata,
but there is no sharing across different engines.
What I'm running because of this¶
The ledger decided the daily driver for me. The dense 27B at TP2 with full 262k context is the fastest thing this board has ever served (167-220 tok/s decode), uses a third of the host RAM the MoE models want, and is the one workload whose bottleneck my Gen4 board can't reach. The MoE models are kept for when the board or the RAM catches up.
Two operational footnotes from the same day, already patched into the compose
files: add --prefix-cache-mib 4096 to the 27B service (its follow-up TTFT was
1.5-2.3 s against Flash-Next's 0.6 s with the flag set), and keep hard
mem_limit caps on every engine container — an OOM that fires inside a cgroup
costs a container, not the machine.
Raw records: ctx-bench-2026-11-08.jsonl (Radiance ladder, one JSON per request)
and the Strata bench-iq2xs-64k report directory (config, results, per-request
outputs) live on the box.