We have argued about GPUs for two years
This week the industry quietly conceded something else: what actually limits AI is often not compute. It is memory.
On 4 August at FMS 2026, SK hynix and SanDisk published the first standard specification for High Bandwidth Flash (HBF), released through the Open Compute Project, so an open standard rather than a proprietary one. The figures in the spec: up to 512GB per stack, roughly 0.4 to 3.0 TB/s, across three grades.
🧠 Why memory is the bottleneck
For every single token a model produces, the weights it needs have to be read out of memory. Not once per sentence, once per token.
So during inference the GPU is frequently not calculating. It is waiting for numbers to arrive.
Why not simply hold more? Because there have been two options and neither fits: HBM is fast but only tens of gigabytes per stack and extremely expensive; SSDs are cheap and large but hundreds of times slower. The middle was empty. HBF aims at exactly that gap, NAND capacity behind something close to an HBM-style interface.
💡 You feel this already
Why is long context expensive? Why is "remembers the whole conversation" a selling point rather than an assumption? Why won't a big model run on your machine?
HBF aims at exactly that gap, NAND capacity behind something close to an HBM-style interface.
Not because it isn't fast enough. Because it doesn't fit.
My own laptop is the plain example, 6GB of video memory, and local vision models simply won't run. Not a shortage of compute. The model doesn't fit.
🎓 The part worth teaching
When we discuss technology, attention lands on the number that is easiest to talk about. GPUs are glamorous, branded, ranked, so GPUs are what everyone argues about.
But the constraint that actually decides whether you can do a thing is often the dull one nobody mentions, sitting one layer down.
That isn't only AI. It's construction too: everyone debates the design, while the thing actually holding the project is a consent, or a trade you cannot hire. Learning to ask "what is genuinely the binding constraint here" is a skill you can practise.
⚠️ What I won't overstate: this is a specification, not a product. Nothing is for sale, there are no benchmarks, and who adopts it and when is unknown. A new tier has been defined, not a problem solved. Standard to faster machine usually takes years.
In your own field, is the thing everyone argues about the same as the thing actually holding you back?
More in AI news
All →- AI newsFraming a house, going from a slab to something with the shape of a building on it,…6 Aug 2026Framing a house, going from a slab to something with the shape of a building on it,…
On a site walk, your hands are already full
Thirty-four US states have issued AI guidance for schools- AI newsThis week Cloudflare started selling to software4 Aug 2026This week Cloudflare started selling to software