Ollama on Apple Silicon: Unified Memory Sizing

Ollama runs on Apple Silicon through the Metal API, per the hardware support documentation, and the memory model is different enough from a discrete card that the usual sizing advice does not transfer cleanly. There is no separate pool of VRAM to fill. The GPU addresses the same memory the operating system uses, which removes the capacity ceiling that dominates discrete-GPU planning and replaces it with a bandwidth ceiling instead.

That trade is the whole story of running local models on a Mac, and it points in a clear direction: Apple Silicon is unusually good at holding large models and unremarkable at running small ones quickly.

Unified memory is not all yours

The headline number on a Mac is total unified memory. The number that matters for model loading is smaller.

Metal exposes an advisory ceiling per device, recommendedMaxWorkingSetSize, described in Apple’s documentation as the working set size above which performance may suffer. On Apple Silicon this ceiling is a fraction of installed memory rather than the whole pool, because macOS has to keep the window server, the file cache, and every running application resident too. In practice the fraction sits near two-thirds on smaller configurations and closer to three-quarters on larger ones.

The planning rule that follows:

Usable model memory ≈ 70 percent of total unified memory

Installed memory	Approximate budget for weights plus cache
16 GB	~11 GB
24 GB	~17 GB
32 GB	~22 GB
48 GB	~34 GB
64 GB	~45 GB
128 GB	~90 GB

The ceiling can be raised with the iogpu.wired_limit_mb sysctl, which lets the GPU wire down more of the pool. That is a real option on a machine dedicated to inference, and a poor one on a laptop you also work on, because the memory comes straight out of what macOS has left. Pushing it too far produces swapping, and swapping to SSD is dramatically worse than the CPU offload it was meant to avoid.

Bandwidth is the ceiling that replaces capacity

On a discrete card, capacity is a hard gate and bandwidth is a dial. On Apple Silicon the gate opens wide and the dial becomes the constraint. Published bandwidth figures from Apple’s technical specifications span more than a factor of six across the current lineup:

Chip	Maximum unified memory	Memory bandwidth
M4	32 GB	~120 GB/s
M4 Pro	64 GB	~273 GB/s
M4 Max	128 GB	~410 or ~546 GB/s
M3 Max	128 GB	~300 or ~400 GB/s
M2 Ultra	192 GB	~800 GB/s
M3 Ultra	512 GB	~819 GB/s

Two rows carry two figures because those chips ship in more than one binning, and the lower core count comes with a narrower memory interface. An M4 Max at 410 GB/s and an M4 Max at 546 GB/s hold identical models and generate at materially different rates. This is the same trap the 16 GB discrete tier sets, described in best GPU for Ollama: matching capacity does not mean matching speed.

What that predicts

Single-stream generation reads the active weight set once per token, so the ceiling is bandwidth divided by weight bytes. Applying that to the Q4_K_M estimate of 0.6 GB per billion parameters:

Chip	8B at Q4	32B at Q4	70B at Q4
M4 (120 GB/s)	~25 tok/s	~6 tok/s, tight at 32 GB	No
M4 Pro (273 GB/s)	~57 tok/s	~14 tok/s	No
M4 Max (546 GB/s)	~114 tok/s	~28 tok/s	~13 tok/s
M3 Ultra (819 GB/s)	~171 tok/s	~43 tok/s	~20 tok/s

Those are theoretical ceilings. Real rates land near 60 to 80 percent of them, as Ollama tokens per second sets out. Even after that discount, the pattern is clear: an M4 Max or an Ultra will run a 70B model at conversational pace, which no single consumer discrete card can do at all.

The weakness: prefill, not generation

Generation is memory-bound and Apple Silicon does well at it. Prompt processing is a different phase with a different bottleneck.

Prefill processes every token of your prompt in parallel, which saturates arithmetic units rather than the memory bus, so it is compute-bound. Apple’s GPUs carry considerably less raw arithmetic throughput than a high-end discrete card of the same era. The consequence is structural rather than incidental: a Mac that generates at a comfortable rate can still take a noticeable pause before it starts, and that pause scales with prompt length.

This matters more than it sounds for the workloads people actually run on large-memory machines. Feeding a long document, a large codebase context, or a retrieval-augmented prompt is exactly the case where prefill dominates. If the plan is short prompts and long answers, Apple Silicon looks excellent. If it is long prompts and short answers, the advantage narrows sharply.

Choosing a configuration

Work backwards from the largest model class you intend to run, then apply the 70 percent rule and add cache headroom. The VRAM calculator gives the weights-plus-cache total for a specific model, quantisation, and context window; divide that by 0.7 to get the unified memory configuration to buy.

Target model class	Weights at Q4_K_M	Minimum sensible unified memory
7-8B	~4.8 GB	16 GB
12-14B	~8.4 GB	24 GB
27-32B	16 to 19 GB	36 to 48 GB
70B	~42 GB	64 GB
70B at Q8, or larger	77 GB and up	128 GB and up

Two observations that change purchase decisions:

Memory is the upgrade that matters, and it cannot be added later. Unified memory is packaged with the chip. A 16 GB machine is a 16 GB machine forever, so the configuration choice at purchase is permanent in a way a desktop’s is not.

Below the 32B class, a discrete card is usually the better buy. A used 24 GB card pairs high bandwidth with enough capacity for the 27-32B class, at a fraction of what a large-memory Mac costs. Apple Silicon earns its premium above that line, where the discrete route requires multiple cards, a power budget, and a case to hold them.

For anyone weighing a small Mac against a modest card for the 7-8B class, the discrete comparison is in best Ollama models for 8GB VRAM, which covers what that tier actually holds.

Practical notes for a Mac

Confirm the split anyway. ollama ps reports the processor split on macOS exactly as it does elsewhere. Unified memory makes partial offload less likely, not impossible, and a machine near its wired limit will still fall back. The diagnostic path in Ollama not using GPU applies.

Watch memory pressure, not free memory. macOS deliberately uses nearly all available memory for caching, so “free memory” is close to meaningless. Activity Monitor’s memory pressure graph is the signal that matters, and yellow during a load means the model is too large for the configuration.

Watch the default context window, which large memory makes bigger. Ollama sizes the default from available memory rather than fixing it, and the context length documentation gives 4K below 24 GiB, 32K from 24 to 48 GiB, and 256K above that. A large-memory Mac therefore lands in the top bucket by default, and a 256K window is an enormous KV cache to allocate and then read on every generated token. This is the one place where a Mac’s abundant capacity works against it: the same setting that is a harmless 4K on an 8 GB card becomes a real memory and bandwidth cost here. Set num_ctx to what the task needs rather than accepting the default, and confirm the allocation in the CONTEXT column of ollama ps.

Quantise the KV cache for long context. OLLAMA_KV_CACHE_TYPE=q8_0 roughly halves cache memory, which on a machine running a 70B model at long context is measured in gigabytes.

Close what you are not using. On a discrete card the desktop competes for a separate 8 or 24 GB pool. On a Mac it competes for the same pool the model is in, so a browser with many tabs is directly subtracting from the model budget.

The summary

Apple Silicon inverts the usual constraint. Capacity, the thing that stops most discrete builds, becomes abundant; bandwidth, which discrete cards have in surplus, becomes the limit. That makes a large-memory Mac the most accessible route to running 70B-class models locally, and a comparatively expensive way to run 8B ones. Size the memory for the largest model you mean to run, check which binning the chip is, and expect prompt processing to be the part that feels slow.