Unified memory, explained properly

One pool, not two

On a PC with a discrete graphics card there are two separate pools of memory. System RAM holds your operating system and applications. Video memory, soldered to the graphics card, holds anything the GPU touches. To run a model on the GPU you must first copy every weight across the PCIe bus into video memory, and if the model is larger than that card you are stuck: 24 GB is 24 GB, no matter how much system RAM is in the machine.

Apple silicon has one pool. The CPU, the GPU and the Neural Engine all address the same physical memory, over the same memory controller, at the same bandwidth. There is no copy step. When MLX loads a model, the weights land in memory once and the GPU reads them where they lie.

What that changes

The practical consequence is that your budget is your whole machine. A 64 GB MacBook Pro can hold a model that no consumer graphics card on the market can hold, because it is not partitioned. This is the single reason Apple silicon became interesting for local inference, and it is why the question "will it fit" is answerable with arithmetic rather than a compatibility matrix.

The less pleasant consequence is that macOS, your browser and your editor are drawing from the same pool. Memory spent on a model is memory the rest of the system does not have.

Where the ceiling really is

Your installed RAM is not your usable RAM. macOS reserves a working set for the kernel, the window server, and whatever else you have open. On this site we assume a tiered reserve: roughly 2 GB on an 8 GB machine, scaling up to around 12 GB on a 256 GB machine, because a Mac Studio owner typically has more open than a Mac mini owner does.

There is a second ceiling. Apple caps how much of unified memory the GPU is allowed to claim — historically around 65 to 75 percent of installed RAM, depending on the machine. You can raise it:

That example gives the GPU 56 GB on a 64 GB machine. It resets on reboot, and if you set it too high your Mac will begin swapping or, in the worst case, become unresponsive. Leave at least 8 GB for the system.

When the budget is exceeded

If a model does not fit, macOS does not refuse to load it. It swaps: pages of weights get written to SSD and read back on demand. Because a decoder reads every weight for every token, swapping turns a model that would have generated 20 tokens per second into one that generates less than one, while writing many gigabytes to your SSD per minute. This is why our fit engine treats "over" as a hard stop rather than a warning. A model that overflows is not slow. It is unusable.

The mental model

Think of it as a shelf of fixed width. macOS takes the left end. The model weights take the largest block. The KV cache grows from the right as your conversation gets longer. Headroom is whatever is left. Everything else on this site is that picture, with real numbers in it.