DeepSeek R1 Distill Qwen 32B
DeepSeek · 32.5B · up to 128K context · mit
Distilled from DeepSeek R1's reasoning traces onto a Qwen2.5 32B base, so on a 32GB Mac you get long chain-of-thought behavior without the 671B parent's memory bill. Expect verbose answers full of visible reasoning steps, which is either useful or annoying depending on the task.
On a M4 Pro with 48 GB
macOS9.6 GBweights18.3 GBKV cache2.1 GBheadroom18.0 GB
- Weights
- 18.3 GB
- KV cache
- 2.1 GB
- Usable RAM
- 38.4 GB
- Headroom
- 18.0 GB
- Generation
- 11.6 tok/sest.
- Prompt processing
- 27.8 tok/sest.
- First token (1K prompt)
- 36.9 sest.
Decode speed is bandwidth ÷ active weight bytes, at 273 GB/s and 78% efficiency. These are modelled figures, not measurements.
Every quantization, on your Mac
| Quant | Weights | + KV | Verdict | Speed est. | Repo |
|---|---|---|---|---|---|
| q4 | 18.3 GB | 20.4 GB | FITS · 18.0 GB FREE | 11.6 tok/s | DeepSeek-R1-Distill-Qwen-32B-4bit |
| q8 | 34.5 GB | 36.7 GB | TIGHT · 1.7 GB FREE | 6.2 tok/s | DeepSeek-R1-Distill-Qwen-32B-8bit |
We never host weights. Every link goes to Hugging Face.
Run it
Chat
mlx_lm.chat --model mlx-community/DeepSeek-R1-Distill-Qwen-32B-4bitServe an OpenAI-compatible endpoint
mlx_lm.server --model mlx-community/DeepSeek-R1-Distill-Qwen-32B-4bit --port 8080How far can you push the context
2K tokensFITS · 19.6 GB FREE
cache 0.5 GB
8K tokensFITS · 18.0 GB FREE
cache 2.1 GB
32K tokensFITS · 11.5 GB FREE
cache 8.6 GB
128K tokensOVER BY 14.2 GB
cache 34.4 GB
Which Macs run this
| Chip | Smallest RAM that fits | Speed est. |
|---|---|---|
| M1 | no configuration | — |
| M1 Pro | 32 GB | 8.3 tok/s |
| M1 Max | 32 GB | 17.3 tok/s |
| M1 Ultra | 64 GB | 35.4 tok/s |
| M2 | no configuration | — |
| M2 Pro | 32 GB | 8.4 tok/s |
| M2 Max | 32 GB | 17.5 tok/s |
| M2 Ultra | 64 GB | 35.9 tok/s |
| M3 | no configuration | — |
| M3 Pro | 36 GB | 6.2 tok/s |
| M3 Max (14-core CPU) | 36 GB | 12.8 tok/s |
| M3 Max (16-core CPU) | 48 GB | 17.5 tok/s |
| M3 Ultra | 96 GB | 37.2 tok/s |
| M4 | 32 GB | 4.9 tok/s |
| M4 Pro | 48 GB | 11.6 tok/s |
| M4 Max (14-core CPU) | 36 GB | 18.2 tok/s |
| M4 Max (16-core CPU) | 48 GB | 24.5 tok/s |
| M5 | 32 GB | 6.3 tok/s |
| M5 Prounverified | 48 GB | 13.3 tok/s |
| M5 Maxunverified | 36 GB | 27.9 tok/s |