The number on the box is a queue-depth number
A DRAM-less NVMe SSD is sold as a 7,100 MB/s drive. It is also a drive with no LPDDR chip on the PCB and no dedicated cache for its flash translation layer. It borrows 64 MB of your system memory through the NVMe Host Memory Buffer feature, and it reads its mapping table out of NAND when that memory is not enough.
I ran a read benchmark on the drive in my own machine. The drive is a Patriot P320 256GB built on a Maxio MAP1202 controller, which PCI-SIG lists as a PCIe 3.0 x4 DRAM-less part. The host is a Ryzen 7 2700X with 30 GB of DDR4. The measured single-flow sequential read came out at 2,029 MB/s. That is 51.5 percent of the theoretical 3.94 GB/s the four lanes could carry, and it is nowhere near the number printed on any DRAM-less retail box.
That gap is not a defect. It is what happens when a workload asks for one read at a time. Every spec sheet in this category is measured at high queue depth, and an agent loop does not run at high queue depth.
What I measured, line by line
Four gigabytes were written to the drive, then read back under four conditions. The first line is the reference point: a memory copy pass on the host, measured with memmove on a 256 MB buffer. A copy touches memory twice, so it is reported as both copy bandwidth and memory traffic.
| Pass | Measured | What it is |
|---|---|---|
| Host DRAM copy, 256 MB buffer | 12,037 MB/s copy / 24,073 MB/s memory traffic | Upper tier. Two passes over memory. |
| Buffered read, page cache warm | 3,157 MB/s | Second read of the same file. Never touches the drive. |
| O_DIRECT sequential read, 1 MB blocks, queue depth 1 | 2,029 MB/s | Bypasses the page cache. This is the drive. |
| Same read split across 4 concurrent flows | 1,942 MB/s aggregate | Queue depth 4. No gain. |
| 20,000 random 4 KB reads, queue depth 1 | 16,049 IOPS, 0.06 ms per read | The mapping-table-sensitive case. |
Two things in that table matter more than the headline number.
The first is that queue depth did not help. Four concurrent readers on the same drive produced slightly less aggregate throughput than one reader. A DRAM-less controller has one ARM core and four NAND channels. At queue depth 1 it is already doing the work it can do. The 7,100 MB/s figure on the retail listing is a queue-depth-32 number, and the same datasheets print a queue-depth-1 random read figure two orders of magnitude lower. Samsung’s own 990 PRO datasheet lists 22,000 IOPS at QD1 against 1,400,000 IOPS at QD32 on the same drive. Those numbers are not in conflict. They describe different workloads.
The second is that the page cache is not free memory. A warm buffered read ran 55 percent faster than the drive itself, because it was reading RAM. On a 30 GB machine running a model that wants its KV cache resident, that is the same RAM the operating system is using for the model. The Host Memory Buffer the drive needs comes out of the same pool.
Why storage is suddenly on the critical path
KV cache offloading moved from a research idea to a design decision because context stopped being short. The Seagate and SK hynix joint white paper on hybrid storage for KV cache offloading states the arithmetic plainly. At the throughput Microsoft demonstrated on a 72-GPU rack, roughly 1.1 million tokens per second, a GPU can generate on the order of 92 TB of KV cache data per day. An H100 with 80 GB of HBM holds about 1.2 minutes of that. A terabyte of CPU DRAM holds about 16 minutes. Sessions that persist for days do not fit in either, so the cache gets evicted and the prompt gets recomputed.
That is the same problem at desk scale. A llama.cpp session that keeps its KV cache in host RAM at 100,000 tokens allocates gigabytes before it generates anything. The DiskLLM fork reports 14,900 MB of private RAM for a 7B model at 256K context, against 258 MB when the KV cache is memory-mapped to an SSD file. Throughput at that point is 2.5 tokens per second, which is what a 2.5 GB machine can actually run. The trade is capacity for speed, and the speed you gave up is the storage tier’s.
For a repeated document the trade usually wins. Micron’s own white paper on on-device AI gives a concrete case: a 10,000-word office policy queried repeatedly costs hundreds of seconds of processor time to regenerate its KV cache per session, against a few seconds to reload it from storage. Its central finding contradicts the spec sheet. Peak throughput does not predict AI responsiveness. Consistent delivery at low queue depth does. That is the column my measurement sits in.
Restore time, computed from the measured rate
Using the per-token KV sizes the Seagate paper publishes, 56 KB per token for Qwen2.5-7B and 320 KB per token for Qwen2.5-72B, here is what restoring a cache costs at three different rates. These are transfer times only. They do not include the CPU work to place blocks into the attention buffers, and they assume the file is already on disk.
| Model / context | KV size | At measured 2,029 MB/s | At spec 7,100 MB/s | At measured DRAM copy 12,037 MB/s |
|---|---|---|---|---|
| Qwen2.5-7B, 10,000 tokens | 0.53 GB | 0.28 s | 0.08 s | 0.05 s |
| Qwen2.5-7B, 100,000 tokens | 5.34 GB | 2.82 s | 0.81 s | 0.48 s |
| Qwen2.5-72B, 10,000 tokens | 3.05 GB | 1.61 s | 0.46 s | 0.27 s |
| Qwen2.5-72B, 100,000 tokens | 30.5 GB | 16.14 s | 4.62 s | 2.72 s |
At the measured rate, a 72B model at 100,000 tokens costs 16 seconds before the first token appears. At the spec-sheet rate it costs 4.6 seconds. Both are real numbers. Only one of them describes a drive being read by one process at a time.
Generation throughput is unaffected either way. The llama.cpp slot-restore tutorial is explicit about this. Prefill per message drops from over a minute to about 0.2 seconds when a saved slot is restored, and token generation stays at whatever the GPU was already doing. Storage changes time to first token. It does not change the decode rate, and the decode rate is what the ceiling on local decode is about.
The cost model in the tooling is calibrated for hard drives
The persistent disk cache fork proxycache-llamacpp has a flag called --cache-disk-read-mbps, default 200, described as the assumed sequential read speed of the cache disk. It is not a cosmetic setting. The fork uses it in a cost model that decides whether a stored state is worth reading from disk or should be kept resident in RAM, with a minimum expected gain of 1,000 ms.
My drive reads at 2,029 MB/s. The default assumption is ten times lower. A tool calibrated for a spinning disk will keep states in RAM that a modern NVMe drive could serve faster than the model can consume them, which on a 30 GB machine is the wrong decision in the wrong direction. The flag exists precisely because the assumption is a guess. Set it to the number you measure, not the number on the box.
What the price move has to do with any of this
NAND contract prices are the reason storage is being discussed at all. A 256 GB TLC SSD contract price reached about $134.20 in the third quarter of 2026 against $33.30 a year earlier, a 303 percent increase. TrendForce expects another 15 to 20 percent quarter over quarter in Q4 2026, with enterprise SSD up 23 to 28 percent, because cloud builders are expanding inference infrastructure. SK hynix has allocated its NAND production through the end of 2026, and Micron says more than 75 percent of expected fiscal 2027 output is already committed. New capacity is not expected to move supply until mid-to-late 2027.
The retail trace is visible on the same listings. The Samsung 990 PRO 1TB, a DRAM-equipped drive carrying 1 GB of LPDDR4, has a tracked low of $59.99 in July 2023 and is listed at about $260 as of October 19, 2026. The Crucial P310 2TB, which is DRAM-less, has a tracked low of $93.99 in July 2025 and is quoted at about $310 as of October 25, 2026. The same pressure is on host memory, which is the resource a DRAM-less drive borrows from. A 16 GB DDR4 desktop kit tracked a low of $26.99 in February 2025 and is at about $130 as of October 14, 2026. That is why what a 3x DRAM price means for local LLM builds stopped being a data center story.
What to actually buy for a KV cache tier
For a local build, the storage tier is not where the interesting money is. A 5.3 GB cache file costs 2.8 seconds to read at my measured rate and 0.5 seconds at DRAM copy rate. If the cache fits in RAM, it stays in RAM and the drive does not matter. The drive matters when the context is long enough that it does not fit, which is the same condition as the memory wall being a cache policy problem before it is a capacity problem.
Three practical positions:
- Short contexts, single session. A DRAM-less drive is fine. The HMB allocation is 64 MB, sequential performance is not materially affected by the missing DRAM, and the price difference is the whole point. The Crucial P310 2TB at about $310 is the same drive class as mine, with a spec sheet number that will never be reached at queue depth 1.
- Long contexts, many sessions, low latency required. Buy the DRAM-equipped drive. The Samsung 990 PRO 1TB carries 1 GB of LPDDR4 for its mapping table, which is 16 times the HMB budget, and its datasheet prints a queue-depth-1 random read figure a DRAM-less part cannot match. The Samsung 990 EVO Plus 1TB, tracked at $289.99 on October 16, 2026 against a $154.99 list price, is the DRAM-less counterexample in the same interface: a headline number within 1 percent of the 990 PRO, and different behavior under sustained random access.
- Host memory is the constraint, not the drive. If you are running a model that wants its KV cache resident, the 64 MB the drive borrows is not the problem. The gigabytes the model wants are. A 16 GB DDR4 desktop kit at about $130 now costs more than a 1 TB NVMe drive did eighteen months ago, and the drive in the second slot does not help you hold a 100,000-token cache in RAM.
For the full build ladder, including where a storage tier fits against GPU memory, see the hardware ladder from budget to enterprise.
Two things I could not measure
Be honest about the boundaries of the numbers above. I could not read the negotiated PCIe link speed for the drive in this environment. The controller is listed by PCI-SIG as a PCIe 3.0 x4 part, and the capability list was not readable from the shell I ran, so the 51.5 percent figure is measured against the Gen3 x4 channel, not against a confirmed link. I also could not query the drive’s Host Memory Buffer state directly, because the nvme CLI is not installed on this host. The module parameter max_host_mem_size_mb reads 128, which is a ceiling, not an allocation.
What is measured is the read rate, the queue-depth behaviour, and the random-read rate on the machine I run on. Those are enough to decide whether a disk tier is worth reading from, which is the decision every cache policy in this space actually makes.


Pingback: The Same SSD Reads at 218 MB/s or 2,349 MB/s: What Block Size Says About the KV Cache Tier - GenX AI Tools
Pingback: Three Cards With the Same 16 GB of Memory: What a $361 to $1,201 Price Spread Says About the Bill of Materials - GenX AI Tools
Pingback: 8.4x at the Median, 29x in Energy: What a Measured Shared KV Pool Says About the Tier That Serves It - GenX AI Tools
Pingback: 21,333 Tokens per Second Is a Batch Number: What a 394.7 TB/s Requirement Says About the Single Stream - GenX AI Tools
Pingback: A Factor of 22 on the Same Card: What KV Cache Placement Says About the Bus - GenX AI Tools