Single dark semiconductor package on a matte surface lit from one side

NVIDIA Quietly Lowered Rubin HBM4 Bandwidth: What a 13 Percent Spec Cut Costs a Decode Loop

NVIDIA did not announce a change to Vera Rubin. The change appeared in the specification table on the product page, visible only by comparing the current page against an archived copy from September 1. Per-GPU HBM4 bandwidth went from 22 TB/s to 19.2. NVLink 6 went from 3.6 TB/s to 3. The rack aggregate went from 1,580 TB/s to 1,400. The eight-GPU HGX configuration took the largest hit: capacity from 2.3 TB to 2 TB, bandwidth from 176 TB/s to 130.

None of that is a marketing adjustment. Memory bandwidth determines how fast a decode loop runs. When a vendor lowers it on paper, the hardware cannot hold the number it promised.

What actually changed

The figures below are NVIDIA’s own, taken from the current product page and from an archived copy of the same page dated September 1, 2026. The revision happened between those dates with no announcement.

Configuration Specification Original Revised Change
Rubin GPU HBM4 capacity 288 GB 288 GB unchanged
Rubin GPU HBM4 bandwidth 22 TB/s 19.2 TB/s -12.7%
Rubin GPU NVLink 6 bandwidth 3.6 TB/s 3 TB/s -16.7%
Vera Rubin Superchip HBM4 bandwidth 44 TB/s 38.5 TB/s -12.5%
Vera Rubin NVL72 HBM4 capacity 20.7 TB 20.7 TB unchanged
Vera Rubin NVL72 HBM4 bandwidth 1,580 TB/s 1,400 TB/s -11.4%
Vera Rubin NVL72 NVLink bandwidth 260 TB/s 216 TB/s -16.9%
HGX Rubin NVL8 HBM4 capacity 2.3 TB 2 TB -13.0%
HGX Rubin NVL8 HBM4 bandwidth 176 TB/s 130 TB/s -26.1%
HGX Rubin NVL8 Networking bandwidth 1.6 TB/s 1.8 TB/s +12.5%

Two details matter more than the percentages. Capacity did not move on the NVL72 configuration; only the rate moved. And scale-out networking went up while the scale-up fabric went down. A vendor trading interconnect bandwidth for network bandwidth is not optimizing for peak throughput. It is optimizing for something that will ship.

NVIDIA’s own documentation disagrees with itself. The HGX page lists the revised system figures while the per-GPU table on the same page still advertises 22 TB/s and 3.6 TB/s. Some regional pages carry the older numbers. That is what a spec revision looks like before it propagates.

Why the numbers moved

The per-pin speed is the mechanism. HBM4 on the Rubin platform was specified at 11.7 Gbps per pin. The revised target is 9.6. Each Rubin GPU integrates eight 36 GB stacks, which is where 288 GB comes from. Eight stacks at 9.4 to 9.6 Gbps per pin land at roughly 19.2 TB/s. Reaching 22 TB/s requires about 10.7 Gbps per pin. The memory makers’ published figures are sufficient for that. NVIDIA’s implementation is not, which points at bus width and signalling on the die rather than at the suppliers.

The same logic runs through the interconnect. The NVLink Nyquist target was lowered from 56 GHz to 47 GHz. A lower Nyquist target reduces electrical demands on the SerDes, the cables, and the dielectric materials in the connector path. It buys signal margin. It costs peak bandwidth. A link that runs at 47 GHz in production beats a link that runs at 56 GHz in a lab and fails at rate.

The stacking change points the same way. Samsung and SK hynix are preparing 8-Hi HBM4 for Rubin Ultra in Q2 2027, a departure from the original 12-Hi HBM4E plan, even though both vendors completed NVIDIA’s qualification for 12-Hi. Twelve-high stacks carry thermal penalties and lower yields. Stack height is a thermal and yield decision, and when yield wins, the capacity plan changes.

What a bandwidth cut costs a decode loop

Decode throughput is bandwidth divided by the bytes a token requires. That is why the 12.7 percent cut on the per-GPU bus, rounded to 13 in this title, is a 12.7 percent cut on the ceiling for a model that reads its weights per token. The bytes per token depend on quantization and on whether the model activates all its parameters for every token.

I measured this on the machine that actually serves the model: a Ryzen 9 5900X with an RTX 3090, 64 GB of DDR4, and a Gen4 NVMe drive. The 3090’s bus is 384-bit at 19.5 Gbps per pin, which TechPowerUp’s GPU database lists as 936.2 GB/s. The engine is Strata, which splits a 125B mixture-of-experts model across GPU, system RAM, CPU, and SSD. The request was sent from a different machine, a Ryzen 7 2700X, so the wall-clock figure includes the network round trip.

The prompt was 1,727 tokens of real text, a fetched specification page, uncached, so the prefill figure is a real prefill.

Measurement Result
Prefill 1,295.0 tokens/s (1,334 ms for 1,727 tokens, cache_n = 0)
Engine-reported decode 117.7 tokens/s (1,700 ms for 200 tokens)
Wall-clock decode through the API 100.9 tokens/s (3.31 s total request)
Draft tokens generated / accepted 149 / 134, 89.9% acceptance

The two decode figures differ by 16.8 tokens/s. Engine timings account for 3.03 of the 3.31 seconds. The remaining 0.28 seconds is request handling, tokenization, and the network round trip. That remainder is what an agent loop pays.

The arithmetic that follows is mine, on a hypothetical per-token read volume, not a published performance figure. 936.2 GB/s divided by 117.7 tokens/s is 7.95 GB per token at full bus utilization. A 125B model at 4 bits is roughly 60 GB of weights. If the engine read all of them per token, that bus would produce about 15 tokens/s. It produced 117.7, so it is reading at most 7.95 GB per token, which is what a mixture-of-experts model does when only a subset of experts activates. The 89.9 percent draft acceptance is the other half: accepted draft tokens skip a full generation pass.

Apply the same arithmetic to the revised Rubin numbers. At 7.95 GB per token, 19.2 TB/s supports about 2,415 tokens/s. At 22 TB/s it would have supported about 2,767. The ratio is exactly the percentage in the spec table. A bandwidth revision is a throughput revision in a different unit.

The same rule holds at every scale. A 273 GB/s bus and a 19.2 TB/s rack differ by a factor of about 70 in bandwidth, and by the same factor in sustainable tokens per second. The mechanism does not change with the size of the bus. Only the number does.

The memory side of the same story

The supply data lines up with the spec revision. SK hynix said on its earnings call that DRAM and HBM production is fully sold out through 2029, and that negotiations with major customers for 2027 HBM volumes and prices are already complete. Micron said on its fiscal fourth-quarter call that more than 75 percent of its fiscal 2027 output is already committed, and that customer discussions have moved to 2028. CEO Sanjay Mehrotra said supply-demand conditions will be tighter in calendar 2027 and 2028 than in 2026, and that demand exceeds supply in both years.

Price is the visible half. Samsung’s average quoted HBM price was tallied at $4.50 per gigabit on October 26, against $4.00 for Micron and $3.60 to $3.80 for SK hynix. TrendForce forecasts a 121 percent blended HBM ASP rise in 2027. The reason a desktop kit costs what it costs is the same wafer competition described in HBM is eating your RAM: HBM and general-purpose DRAM draw on the same advanced process and the same wafer starts, and HBM takes priority.

A concrete number makes it readable. The G.SKILL Flare X5 64 GB kit, 2×32 GB DDR5-6000 CL30 UDIMM, carries a list price of $194.99. Price history for that SKU, tracked since October 18, 2023, records a low of $187.00 on February 28, 2024 and a high of $1,299.99 on October 16, 2026. It is listed at $1,279.99 today, about 6.5 times the list price. Same module, same SKU, same timings. Nothing about the module changed. The allocation did.

What to buy if you are running models locally

A local build is not on the same fabric, so the revision does not change what to buy. It changes what to expect from a vendor page.

For a card, the relevant figure is the bus, not the model name. A 256-bit GDDR7 card at 30 Gbps per pin delivers 960 GB/s, which is what TechPowerUp’s GPU database lists for the 5080. A 512-bit card at 28 Gbps delivers 1,792 GB/s. That is the entire difference between the tiers a reader can actually buy, and it is the ratio that bandwidth per dollar is computed from. The ASUS TUF RTX 5080 has no active buybox price today. The listing resolves, but it shows no offer, so the only honest number is the record: price history for that SKU, tracked since January 30, 2025, lists a manufacturer price of $1,699.99, a current Amazon-side record of $1,899.99 as of October 15, 2026, and a lowest recorded Amazon price of $1,199.99 on November 20, 2025. A card that has moved between $1,199.99 and $1,899.99 on the same listing is the volatility you are buying, and it is why a price-history check matters more than a launch price.

For system RAM, the tier is decided by whether the model fits before offloading. A 64 GB kit at $1,279.99 is roughly where a 30B model at 4 bits stays resident instead of paging. That is not a comfort purchase. Every offload trip per token is a round trip to a slower tier, and the decode rate is the sum of those trips.

For the storage tier, the drive is part of the inference path when a lookup table is read per token. A Gen5 drive rated by Crucial at 14,900 MB/s read is listed at $400.46 today. Against a Gen4 drive in the same capacity range, the difference is queue depth on random reads, which is what a per-token expert lookup actually looks like.

The practical read

A spec sheet is a forecast. A forecast revised quietly is a forecast that failed against hardware, and the revision shows where the constraint was: signalling, stack height, yield. Those are not solved by buying a bigger card. They are the reason the numbers on the card change. Treat published bandwidth as an upper bound vendors are allowed to lower, and measure the rate you actually get through the API. The engine-reported number and the wall-clock number are not the same number. The difference between them is yours.

Products for this workload

1 Comment

Leave a Reply

Your email address will not be published. Required fields are marked *