Llama 3.3 70B Instruct
Meta · 70.6B · up to 128K context · llama-3.3
Meta tuned this to match Llama 3.1 405B on many benchmarks at a sixth of the size, and a 64GB Mac is the realistic floor for the 4-bit build. It's a strong generalist but the 80-layer depth makes it noticeably slower token-per-second than the 32B tier on the same hardware.
On a M4 Pro with 48 GB
macOS9.6 GBweights39.7 GBKV cache2.7 GBover4.0 GB
- Weights
- 39.7 GB
- KV cache
- 2.7 GB
- Usable RAM
- 38.4 GB
- Headroom
- 0.0 GB
- Generation
- 5.4 tok/sest.
- Prompt processing
- 12.8 tok/sest.
- First token (1K prompt)
- 80.1 sest.
Decode speed is bandwidth ÷ active weight bytes, at 273 GB/s and 78% efficiency. These are modelled figures, not measurements.
Every quantization, on your Mac
| Quant | Weights | + KV | Verdict | Speed est. | Repo |
|---|---|---|---|---|---|
| q4 | 39.7 GB | 42.4 GB | OVER BY 4.0 GB | 5.4 tok/s | Llama-3.3-70B-Instruct-4bit |
| q8 | 75.0 GB | 77.7 GB | OVER BY 39.3 GB | 2.8 tok/s | Llama-3.3-70B-Instruct-8bit |
We never host weights. Every link goes to Hugging Face.
Run it
Chat
mlx_lm.chat --model mlx-community/Llama-3.3-70B-Instruct-4bitServe an OpenAI-compatible endpoint
mlx_lm.server --model mlx-community/Llama-3.3-70B-Instruct-4bit --port 8080How far can you push the context
2K tokensOVER BY 2.0 GB
cache 0.7 GB
8K tokensOVER BY 4.0 GB
cache 2.7 GB
32K tokensOVER BY 12.0 GB
cache 10.7 GB
128K tokensOVER BY 44.3 GB
cache 42.9 GB
Which Macs run this
| Chip | Smallest RAM that fits | Speed est. |
|---|---|---|
| M1 | no configuration | — |
| M1 Pro | no configuration | — |
| M1 Max | 64 GB | 8.0 tok/s |
| M1 Ultra | 64 GB | 16.3 tok/s |
| M2 | no configuration | — |
| M2 Pro | no configuration | — |
| M2 Max | 64 GB | 8.1 tok/s |
| M2 Ultra | 64 GB | 16.5 tok/s |
| M3 | no configuration | — |
| M3 Pro | no configuration | — |
| M3 Max (14-core CPU) | no configuration | — |
| M3 Max (16-core CPU) | 64 GB | 8.1 tok/s |
| M3 Ultra | 96 GB | 17.1 tok/s |
| M4 | no configuration | — |
| M4 Pro | 64 GB | 5.4 tok/s |
| M4 Max (14-core CPU) | no configuration | — |
| M4 Max (16-core CPU) | 64 GB | 11.3 tok/s |
| M5 | no configuration | — |
| M5 Prounverified | 64 GB | 6.1 tok/s |
| M5 Maxunverified | 64 GB | 12.8 tok/s |