A desktop machine can now hold a 300-billion-parameter model
A desktop machine can now hold a 300-billion-parameter model. Holding it is not the same as running it well.
AMD opened its IFA keynote in Berlin on Friday with the Ryzen AI Max PRO 400, codenamed Gorgon Halo. The pitch is a single number: 192GB of unified memory shared by the CPU and GPU, with up to 160GB of that pool assignable to the integrated graphics as VRAM. AMD's claim is that this is enough to run models above 300 billion parameters on one machine with nothing plugged into the cloud. HP and Lenovo are the launch partners, shipping this quarter.
This one isn't really news to me. It's a sum I have been redoing for about a year.
I have wanted a local AI at home that can actually carry real work. Not out of gadget appetite: some of what I handle at work carries a duty of confidence, and there is material I would rather never left the machine. Every time I ran the numbers the answer came back the same. The box that could hold a large model was not a desktop, it was a workstation I would have to think hard before buying, and the alternative was accepting a much smaller model. Capacity was the gate.
This week the gate opened halfway. The other half is worth saying out loud.
I have wanted a local AI at home that can actually carry real work.
At four-bit quantisation a model costs roughly half a gigabyte per billion parameters, so 300 billion is about 150GB of weights. This chip runs four channels of LPDDR5X-8533, a theoretical 273GB per second. A dense model has to read all 150GB again for every single token it produces. That works out at about 1.8 tokens a second, and that is the ceiling, not the experience. It will answer you. You will watch it type.
A mixture-of-experts model changes the arithmetic completely. If only 30 billion of those 300 billion are active per token, the same sum lands in the low tens of tokens per second, which is genuinely usable. So "runs 300B" is not an overstatement. Whether it runs well depends on which model you load, not on the headline.
What I take from it is this. For years the only question was whether my machine was big enough. Capacity is now being solved, and the bottleneck has moved to bandwidth. The number that matters next is not how many gigabytes, it is how many gigabytes per second.
And in the end I am not waiting for this machine because it is fast. Fast has been available in the cloud all along, and cheaply. I am waiting for the version where you can pull the network cable out and it keeps working.
