M5 Max vs M5 Ultra: sizing memory for local LLMs

Choosing between Mac Studio configurations for running language models comes down to two questions: will my model fit, and how fast will it run. Memory size answers the first; memory bandwidth answers the second. This post gives you the arithmetic for both, then applies it to three configurations:

The last row matters more than the first: by default macOS lets the GPU use roughly three quarters of the memory, keeping the rest for the system. That's your practical budget for a model.

Step 1: the weights#

A model's weights take its parameter count times the bytes per parameter:

The quantised figures include the scales MLX stores alongside the weights (about half a bit per weight with its default group size of 64). So:

• Qwen3-32B (32.8 billion parameters) at 4-bit: 32.8 × 0.56 ≈ 18 GB
• Llama 3.3 70B (70.6 billion) at 8-bit: 70.6 × 1.06 ≈ 75 GB
• Qwen3-235B-A22B (235 billion) at 4-bit: 235 × 0.56 ≈ 132 GB

Step 2: the KV cache#

The model also stores a key and a value for every token of context in every layer. Per token:

KV bytes = 2 × layers × KV heads × head dimension × 2 (for 16-bit values)

You'll find the numbers in the model's config.json as num_hidden_layers, num_key_value_heads and head_dim (or hidden_size ÷ num_attention_heads).

All with a head dimension of 128. Qwen3's native context is 32K tokens; longer contexts need rope scaling.

Two things jump out. First, at long contexts the cache rivals the weights. Second, the cache is per conversation: serving four long conversations at once needs four caches.

Step 3: add headroom#

Leave a margin of a few gigabytes inside the GPU budget for runtime buffers and activations, more for long prompts. Then:

weights + KV cache + a few GB ≤ GPU budget

What fits where#

Putting the three steps together, with a 32K context unless noted:

"Tight" means it fits with only a few gigabytes of margin: fine for a single short conversation, but you'll want a shorter context or a smaller quantisation.

The very largest open models — DeepSeek's 671-billion-parameter models, for example, around 380 GB at 4 bits — don't fit in any single Mac Studio configuration here.

Raising the GPU limit#

The three-quarters default is conservative on large machines. On macOS you can raise it with sysctl iogpu.wired_limit_mb if you have administrator rights, trading memory the system would otherwise use. It helps with "tight" models, but leave the operating system a healthy margin — 8 to 16 GB — or the machine will start swapping, which is far slower than any GPU.

Step 4: how fast#

For generation, the GPU reads every active weight for every token, so:

tokens per second ≤ bandwidth ÷ bytes read per token

Ceilings, not benchmarks: expect about 60–75% of them in practice.

The M5 Ultra's doubled bandwidth roughly doubles generation speed for the same model. For a 70B model, that's the difference between reading along comfortably and waiting.

Mixture-of-experts models only read their active experts per token. Qwen3-235B-A22B activates about 22 billion of its 235 billion parameters, so on the 256 GB Ultra it generates much faster than its size suggests — the whole model must fit, but each token only touches a slice.

Prompt processing is compute-bound instead, and scales with GPU cores. The M5 Ultra has more of them than the M5 Max, so long prompts start answering sooner too.

Which to choose#

Mac Studio M5 Max, 128 GB — the all-rounder. Everything up to 70B at 4-bit with long contexts, 30B-class models at 8-bit, several smaller models side by side. Reasonable speed on 70B, fast on 30B and below.

Mac Studio M5 Ultra, 96 GB — speed over size. Twice the bandwidth of the M5 Max for models up to about 60 GB including context: 70B at 4-bit with a moderate context, 32B at 8-bit. Choose it when tokens per second matter more than the largest model.

Mac Studio M5 Ultra, 256 GB — the big one. 70B at 8-bit with room for long contexts, 100–200-billion-parameter mixture-of-experts models such as Qwen3-235B-A22B at 4-bit, or several large models at once.

Test before you commit#

Paper maths gets you close, but the real test is your model, your prompts and your context length. Run mlx_lm.generate and read its peak memory and tokens-per-second figures, as shown in Running Qwen3 and Llama with MLX.

The M5 Max is available to reserve by the day now; the M5 Ultra configurations are coming soon and listed on the pricing page.