M3 Ultra · 512 GB
819 GB/s memory bandwidth · 435.2 GB usable for models · 76.8 GB left to macOS
The short answer
38 models run comfortably and 1 more will run tight at 8K context. The largest that fits is Qwen3 235B-A22B at q8, generating around 29.1 tokens per second.
macOS76.8 GBweights249.7 GBKV cache1.6 GBheadroom183.9 GB
Every model on this Mac
| Model | Size | Best quant | Verdict | Speed est. |
|---|---|---|---|---|
| Qwen2.5 0.5B Instruct | 490M | bf16 | FITS · 434.1 GB FREE | 694 tok/s |
| BGE-M3 | 567M | bf16 | FITS · 434.1 GB FREE | 599 tok/s |
| Qwen3 0.6B | 600M | bf16 | FITS · 433.1 GB FREE | 566 tok/s |
| Parakeet TDT 0.6B v2 | 600M | bf16 | FITS · 434.0 GB FREE | 566 tok/s |
| Llama 3.2 1B Instruct | 1.2B | bf16 | FITS · 432.5 GB FREE | 274 tok/s |
| Whisper Large v3 | 1.6B | bf16 | FITS · 432.1 GB FREE | 219 tok/s |
| Qwen3 1.7B | 1.7B | bf16 | FITS · 430.9 GB FREE | 200 tok/s |
| Llama 3.2 3B Instruct | 3.2B | bf16 | FITS · 428.3 GB FREE | 106 tok/s |
| Qwen3 4B | 4B | q8 | FITS · 429.7 GB FREE | 160 tok/s |
| Qwen3 Embedding 4B | 4B | q8 | FITS · 429.7 GB FREE | 160 tok/s |
| Mistral 7B Instruct v0.3 | 7.3B | q8 | FITS · 426.4 GB FREE | 87.6 tok/s |
| Qwen2.5 7B Instruct | 7.6B | bf16 | FITS · 419.5 GB FREE | 44.7 tok/s |
| InternVL3 8B | 8B | q8 | FITS · 426.7 GB FREE | 80.0 tok/s |
| Llama 3.1 8B Instruct | 8.0B | bf16 | FITS · 418.1 GB FREE | 42.3 tok/s |
| Qwen3 8B | 8.2B | q8 | FITS · 425.3 GB FREE | 78.0 tok/s |
| Qwen2.5-VL 7B Instruct | 8.3B | q8 | FITS · 425.9 GB FREE | 77.1 tok/s |
| Gemma 2 9B IT | 9.2B | q8 | FITS · 425.4 GB FREE | 69.2 tok/s |
| Llama 3.2 11B Vision Instruct | 10.7B | bf16 | FITS · 413.8 GB FREE | 31.8 tok/s |
| Qwen2.5 14B Instruct | 14.7B | q8 | FITS · 418.0 GB FREE | 43.5 tok/s |
| Qwen3 14B | 14.8B | q8 | FITS · 418.1 GB FREE | 43.2 tok/s |
| GPT-OSS 20B | 21B | q8 | FITS · 412.5 GB FREE | 178 tok/s |
| Codestral 22B v0.1 | 22.2B | q8 | FITS · 409.7 GB FREE | 28.8 tok/s |
| Mistral Small 24B Instruct 2501 | 24B | q8 | FITS · 408.4 GB FREE | 26.7 tok/s |
| Gemma 3 27B IT | 27B | bf16 | FITS · 381.2 GB FREE | 12.6 tok/s |
| Gemma 2 27B IT | 27.2B | q8 | FITS · 406.3 GB FREE | 23.5 tok/s |
| Qwen3 30B-A3B Instruct 2507 | 30.5B | q8 | FITS · 402.0 GB FREE | 194 tok/s |
| Qwen3 Coder 30B-A3B Instruct | 30.5B | q8 | FITS · 402.0 GB FREE | 194 tok/s |
| Qwen2.5 32B Instruct | 32.5B | q8 | FITS · 398.5 GB FREE | 19.7 tok/s |
| Qwen2.5 Coder 32B Instruct | 32.5B | q8 | FITS · 398.5 GB FREE | 19.7 tok/s |
| DeepSeek R1 Distill Qwen 32B | 32.5B | q8 | FITS · 398.5 GB FREE | 19.7 tok/s |
| Qwen3 32B | 32.8B | q8 | FITS · 398.2 GB FREE | 19.5 tok/s |
| Qwen2.5-VL 32B Instruct | 33B | q8 | FITS · 398.0 GB FREE | 19.4 tok/s |
| Mixtral 8x7B Instruct v0.1 | 46.7B | q8 | FITS · 384.5 GB FREE | 49.6 tok/s |
| Llama 3.3 70B Instruct | 70.6B | q8 | FITS · 357.5 GB FREE | 9.1 tok/s |
| Qwen2.5 72B Instruct | 72.7B | q8 | FITS · 355.3 GB FREE | 8.8 tok/s |
| Llama 3.2 90B Vision Instruct | 88B | q4 | FITS · 385.7 GB FREE | 13.7 tok/s |
| GPT-OSS 120B | 116.8B | q8 | FITS · 310.5 GB FREE | 125 tok/s |
| Qwen3 235B-A22B | 235B | q8 | FITS · 183.9 GB FREE | 29.1 tok/s |
| DeepSeek V3 | 671B | q4 | TIGHT · 57.8 GB FREE | 32.7 tok/s |