Bandwidth Per Dollar: What a $7,000 Graphics Card Says About Memory Allocation

What the price data actually shows

The RTX 5090 launched at $1,999 in January 2025. Tom’s Hardware’s price tracker, last updated October 8, 2026, lists the best current US price for that card at $7,177. A separate October 7 report on a prebuilt system priced the card itself near $7,000, which is why a complete machine at $7,499 came in cheaper than buying that card plus a $690 CPU separately. Third-party marketplace listings have run as high as $9,500 while first-party stock disappeared from US retail.

That is not a shortage of silicon. Silicon Analysts’ September 2026 analysis of the discrete GPU market is explicit about the difference between this cycle and the 2020 to 2022 one: the binding constraint moved from logic die fabrication to GDDR7 memory allocation. The same firm’s September 7 piece puts GDDR6 prices up roughly 30 percent and broader DRAM pricing up about 170 percent year over year, with memory now the dominant bill-of-materials line on many cards rather than the GPU die.

The volume data makes the picture stranger. Jon Peddie Research recorded a four-year record for discrete GPU shipments in 2026, with notebook GPU units up roughly 16.8 percent, while desktop supply was deliberately rationed. Nvidia was reported in December 2025 planning a 30 to 40 percent cut to RTX 50 series supply for the first half of 2026, prioritising memory allocation toward data-centre products. That is a capital allocation decision, not a demand signal. Shipments of roughly 12.5 million units in tracked periods, with Q2 2026 up about 14 percent year over year, ran alongside record prices.

Nothing here contradicts itself. The same memory that carries an HBM stack in a data-centre accelerator competes for wafer capacity with the GDDR on a consumer card, and the higher-margin product wins the allocation. Nvidia holds roughly 90 percent of discrete add-in-board shipments, so there is very little competitive buffering when that allocation shifts.

Why memory, not the die, sets the price

High Bandwidth Memory consumes about three times the wafer area per gigabyte that standard DDR5 does, because it stacks dies vertically and connects them through silicon rather than laying them out flat. In 2026 HBM accounts for roughly 23 percent of total DRAM wafer output, up from about 19 percent a year earlier. HBM capacity is growing around 29 percent annually while commodity DRAM capacity grows around 10 percent. Those two growth rates cannot both be satisfied from the same wafer starts.

Contract prices for LPDDR5X climbed 89 percent in the second quarter of 2026. Intel CPU prices have climbed roughly 30 percent, with a further 10 percent round reported on October 5, partly because Intel is diverting its own fabrication capacity toward higher-margin Xeon server parts, which gives competitive cover for the increase. Server DDR4 now sells at two to three times its November 2022 level and server DDR5 at nine to thirteen times, according to the DRAM price analysis covering the same market.

For a buyer, the practical effect is simple. The memory on the card is now the expensive part of the card, and the memory in your motherboard is priced by the same market. A 32 GB DDR4 desktop kit, two 16 GB UDIMMs, costs about $251 today. That is not a rounding error on a build any more.

What this changes for a local build

Single-stream decode throughput is bounded by memory bandwidth divided by the bytes read per token. For a dense model that is the whole weight file. For a mixture-of-experts model it is only the active experts, which is why a 125B MoE can run comfortably inside 12 GB of VRAM at 90 to 120 tokens per second on the same card that serves a dense 27B at Q5 around 25 tokens per second. Both numbers are measurements from my own machine, not vendor claims, and the gap between them is entirely explained by how much memory each token actually reads. The engine behind that MoE is covered in the write-up on it.

So capacity per dollar is the wrong metric. Bandwidth per dollar is the metric that determines whether a model is usable or merely fits. A card that holds your model at 320 GB/s will sit at 320 GB/s no matter how much compute it has, because decode is a memory problem. This is the same reasoning that decides whether a model runs on GPU at all or spills to system RAM, where a dual-channel DDR4-2667 desktop delivers about 42 GB/s theoretical and roughly 33 GB/s sustained. That is a twenty-fold difference from a 384-bit GDDR6X bus.

The ladder as it prices today

Prices below are listings I checked today, not MSRP. Bandwidth figures are published specifications.

Read the last two columns separately, because they answer different questions. The RX 9060 XT wins on capacity per dollar: 16 GB for $520 is the cheapest way to hold a 12 to 16 GB model. It loses on speed, and it loses by a lot. The RX 9070 GRE gives you 4 GB less memory and 35 percent more bandwidth for fifty dollars more. If the model fits, the GRE is the better machine. The RX 9070 at $699.99 is where the bus actually steps up to 256-bit, and that is the first tier where decode speed stops being a compromise.

The used RTX 3090 is the interesting case. At a going rate of about $1,622 across 82 active listings, it is the cheapest way to get a 384-bit bus and 936 GB/s. On capacity per dollar it is worse than three cards half its price. On bandwidth per dollar it is also worse than the RX 9070. It is worth buying for the bus, not for the memory size, and the price has moved from $1,025 to $1,500 on the same tracker inside a month. That movement is the memory market reaching second-hand hardware.

The RTX 5090 is not a local-LLM purchase. It is a data-centre product sold through a gaming channel, and the buyers setting its price are the ones buying by pallet. If you need 1,792 GB/s, you are buying a rack, not a card.

What I would actually buy under these prices

Under $600, the GIGABYTE RX 9070 GRE 12GB at $569.99, on the assumption that your model fits in 12 GB. The 192-bit bus is the reason. If it does not fit, the ASRock RX 9060 XT 16GB at $519.99 holds more, and you accept 320 GB/s as the ceiling. That is a real trade, not a preference: a 128-bit bus is a permanent limit.

Under $700, the ASRock RX 9070 16GB at $699.99 is the tier where bandwidth stops being the constraint you have to write around. Above that, the used 3090 remains the value pick for a dense model that needs both capacity and a wide bus, and the hardware tier guide at the pillar article covers where those builds sit against newer generations.

System RAM is the part people forget when the memory market tightens. My own machine runs two 32 GB DDR4-2667 modules, which is a 42 GB/s theoretical ceiling against a 936 GB/s GPU bus. If you are offloading layers to system RAM, you are paying the memory market twice: once for the card and once for the sticks. A 32 GB kit at about $251, such as the Crucial 2×16 GB UDIMM kit, is not expensive in absolute terms. It is expensive relative to what the same kit cost four years ago, and that difference is the whole story of this cycle.

The measurement gap nobody puts in the benchmark

Engine-reported throughput and wall-clock throughput through the API are different numbers, and an agent loop experiences the second one. On my machine today, serving a 125B MoE through llama.cpp, the engine reported 95 to 99 tokens per second. Warm consecutive requests showed 0.17 to 0.18 seconds of overhead, so wall-clock matched the engine figure closely: 73 tokens per second on a 176-token completion.

Short completions are where the gap appears. A 34-token completion measured 98.4 tokens per second engine-side and 1.5 tokens per second wall-clock, because 21.4 seconds of that request was connection setup, tokenisation and scheduler queue rather than inference. Cold-path requests at the start of a session showed 22 to 26 seconds of wall time against under four seconds of engine work. If you are running an agent that makes many short calls, the fixed per-request overhead dominates the token count, and a benchmark quoted as tokens per second tells you nothing about your actual loop. The backend choice changes how much of that overhead exists, which is the subject of the backend comparison.

Bottom line

Memory allocation, not fabrication capacity, is setting GPU prices through at least mid-2027. That means buying on bandwidth per dollar rather than capacity per dollar, and it means the used market for older wide-bus cards is being repriced by the same demand that fills HBM stacks. The cheapest card that fits your model is not the cheapest card that runs it, and under these prices the difference is a full memory bus.