A dark server room corridor with one closed cabinet

A 640 GB/s Card on a 7 GB/s Link: What Lucebox Put the Expert Split On

Lucebox has opened direct checkout on a $5,999 workstation that pairs two AMD memory systems with a 2.3x bandwidth gap. It is unusual for one reason that matters more than the price: it puts a discrete graphics card on a platform where every other vendor skipped one.

The published benchmark claims 51.1 tokens per second on a 284-billion-parameter mixture-of-experts model, against 14.09 tokens per second on a single NVIDIA DGX Spark. That is a 3.63x ratio. The company also publishes a matched context sweep averaging 47.75 against 14.09, which is 3.39x. Both numbers are on the same page. The second is the honest one.

What makes the result interesting is not the ratio. It is the mechanism, and the mechanism has a weak point the marketing does not foreground: the two GPUs are joined by PCIe 4.0 x4, which the company’s own spec sheet puts at roughly 7 to 8 GB/s in practice. A fast card and a slow link. The whole design is an attempt to keep the link idle.

Why an even split fails, and what replaced it

Tensor parallelism divides the same calculation evenly between matching devices. These devices are not matching. The Radeon AI PRO R9700 has 32 GB of GDDR6 at 640 GB/s; the APU side has 128 GB of LPDDR5X-8533 at 273 GB/s. An even split leaves the faster device waiting for the slower one, so the effective rate is the slower rate.

Pipeline parallelism splits the model into whole layers. That works on any architecture, and it costs one cross-device hop per token. On a 7 to 8 GB/s link, that hop is the bottleneck for anything large enough to be a weight tensor.

Activations are a different size. A hidden state of a few kilobytes crosses the link in about half a microsecond, against a 65 millisecond step. The arithmetic decides the design: at 8 GB/s, one gigabyte across the link takes 125 milliseconds, and Lucebox’s own step timings put the full-model step at 64.8 to 65.1 milliseconds. Any weight crossing per token is roughly twice the whole step. Weights must be resident on the device that runs them. Only results cross.

Mixture-of-experts models make that possible. Each layer in the tested model holds 256 expert blocks and only four run per token. If each expert is permanently assigned to one device, the hot experts sit next to the dense path on the fast card and the long tail lives in the larger pool. Both devices run the experts they own at the same time, and the fast card combines the results. No weight crosses.

The two memory pools are not one 160 GB pool. They are two separate pools with a placement policy between them. That policy is the product.

What I measured on my own hardware

My build is the cheap version of the same architecture: a discrete card with a fast pool, system memory with a slow pool, and a model split across them. The serving box is an AMD Ryzen 9 5900X with 64 GB of DDR4 and an RTX 3090 with 24 GB of GDDR6. The engine is Strata, the inference engine that runs a 125B MoE on a Ryzen 5900X and an RTX 3090. The gap here is far worse than Lucebox’s: the card’s published spec is 936 GB/s against 42.7 GB/s for dual-channel DDR4-2667, a factor of about 22 against Lucebox’s 2.3.

I ran a real request through the API from a different machine, a Ryzen 7 2700X, so the wall-clock figures include the network round trip. The engine reports its own timings separately. Both are printed here, because the difference between them is what an agent loop experiences.

Request Prompt tokens Output tokens Engine-reported decode Wall-clock rate Draft accepted
Long prompt 5,388 128 91.8 tok/s 32.1 tok/s 70 of 102 (68.6%)
Short prompt 60 40 107.2 tok/s 48.2 tok/s 26 of 31 (83.9%)

The gap on the long request is not network overhead. The engine spent 2,556.7 milliseconds on prefill and 1,393.7 milliseconds generating, which is 3.95 seconds of the 3.99-second wall. Request and network overhead was about 40 milliseconds. The wall-clock rate looks bad because it includes prefill, and a long prompt is the normal shape of an agent call.

A third request, sent after the server had been idle, showed the other half of the gap. The engine reported 99.0 tokens per second on a 200-token generation. The wall clock took 27.63 seconds for the same call, against 512.7 milliseconds of prompt processing and 2,021.1 milliseconds of generation in its own accounting. About 25 seconds of that wall was not inference. An idle server charges the first call for state a warm server already has.

Draft acceptance is the number worth watching. Lucebox reports a 0.97 acceptance rate across its matched context sweep and full acceptance on the 53-token ceiling test that produced its 55 tokens per second. My measured acceptance was 68.6 percent on the long prompt. Speculative decoding gains depend on the draft being accepted. A benchmark at near-perfect acceptance is a ceiling, not a serving rate.

The tiers, measured on the serving box

I ran a copy pass on the serving box itself over SSH, five passes per size, using a memory move rather than a Python read. A copy touches memory twice, so every figure below is memory traffic and the data-moved share is half of it. Rows under 1 MB never reach DRAM, so they are rates inside cache, not shares of the channel.

Buffer Median copy bandwidth (GB/s) Best pass What it touches
64 KB 40.96 50.41 inside cache, no DRAM traffic
256 KB 55.78 56.99 inside cache, no DRAM traffic
1 MB 20.64 20.85 DRAM traffic, 48.3% of the published channel as traffic, 24.2% as data moved
4 MB 20.81 21.11 DRAM traffic, 48.7% / 24.4%
16 MB 20.91 21.20 DRAM traffic, 49.0% / 24.5%
64 MB 18.28 18.54 DRAM traffic, 42.8% / 21.4%
128 MB 17.17 17.80 DRAM traffic, 40.2% / 20.1%
256 MB 14.06 14.82 DRAM traffic, 32.9% / 16.5%

The channel figure is the published dual-channel DDR4-2667 number, 42.7 GB/s. The board holds two 32 GB DIMMs; the two-channel count is assumed from the platform, not read from the machine. A second independent run, five passes per size, agreed with the table on every DRAM row except 16 MB, where the median fell from 20.91 to 14.87. That row sits at the boundary where the working set stops fitting in the processor’s L3, so it is the least stable number here. Read the DRAM tier as a range of roughly 14 to 21 GB/s, not as one figure.

The practical reading: the DRAM tier settles at about 20 GB/s of memory traffic, roughly half the published channel, and falls as the working set grows past the cache. A model whose hot weights sit in that tier runs at that rate no matter how fast the card is. This is the same ceiling as the 273 GB/s bus, approached from the cheap end.

Where the claim stops working

Lucebox states the limitation plainly: the advantage is structurally tied to mixture-of-experts architectures. A dense model reads every weight for every token. There is no hot set to place on the fast card, so the split buys nothing and the slower pool sets the pace.

Three caveats belong on the record. The benchmark is self-reported by the vendor. The two sides ran different quantizations, a compressed build on one machine and a Q2 build on the other, each on its own tuned software. It is a whole-system comparison, not a controlled A/B test of two GPUs.

Public third-party results bracket the claim rather than confirm it. Two DGX Sparks without a helper model reach 45.7 tokens per second on the same model; two with the same draft model reach 65.09. A single box at 51.1 sits between them. The buyer’s question is not the ratio against one card. It is whether one machine at $5,999 is the right way to get 50 tokens per second, or two machines at $4,699 each with a faster interconnect.

The interconnect is the part a buyer should read carefully. A 100 GbE option is listed at $799 because the internal link is not enough for cluster work. Two boxes joined by one cable run DeepSeek V4.1 Flash, a 383 GB model, at up to 103 tokens per second, reported as 1.9x a single box. That is a different model from the 284B benchmark above, so it is not a like-for-like number. It shows where the gain comes from: more fast memory on the fast side, and a link that is not 7 GB/s.

What this means for a local build

The decision is not which card is faster. It is which weights sit where and how many have to move per token. If a model fits inside 32 GB of GDDR6, the card is pure bandwidth and there is no placement problem. If it does not, you are buying a policy and a slow link, and the policy only pays on a sparse model.

On a dense model, a 32 GB card plus 128 GB of unified memory at 273 GB/s is a 273 GB/s machine with an expensive card attached. On a sparse model with four experts per token, the same machine is a 640 GB/s machine for the part that runs every token and a 273 GB/s machine for the rest. That is the whole argument, and it only holds for the second case.

Prices as of November 2, 2026: the ASRock Radeon AI PRO R9700 Creator 32GB lists at $1,839.99 at this listing, the PowerColor R9700 32GB at $1,649.96 at this listing. Both are 256-bit cards on the same 640 GB/s spec. The 32 GB tier is the constraint, and the allocation logic behind it is in what die allocation says about the 32 GB tier.

If the model is dense, the memory kit matters more than the card. A 64 GB desktop UDIMM kit at DDR5-6000 gives a two-channel board about 96 GB/s as published, against the 42.7 GB/s I measured on the DDR4-2667 board. The G.SKILL Flare X5 64GB (2x32GB) CL30 kit is this listing, $1,299.99 on November 2, 2026 and currently unavailable. The price spread between memory and cards is bandwidth per dollar.

The order situation matters before deciding. The tested machine is the 128 GB configuration. The current line is listed with up to 192 GB of unified memory at $5,999 for the first 200 units against a $6,999 base, with January 2027 deliveries. Batch 3 of the earlier Strix Halo system sold out in late September 2026; Batch 4 ships in November 2026, and the company warns the price will increase. A benchmark published by a vendor selling the machine it benchmarked is a marketing document with numbers in it. The mechanism is sound; the ratio is not independently reproduced.

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *