Mixtral 8x7B Instruct v0.1
Mistral AI · 46.7B (12.9B active, MoE) · up to 32K context · apache-2.0
One of the earliest MoE models to run well on Apple Silicon, needing a 32GB Mac for 4-bit despite having only 12.9B active parameters per token. It still runs faster than a dense 46B model would, but newer MoE designs like Qwen3-30B-A3B have mostly superseded it on quality.
On a M4 Pro with 48 GB
macOS9.6 GBweights26.3 GBKV cache1.1 GBheadroom11.1 GB
- Weights
- 26.3 GB
- KV cache
- 1.1 GB
- Usable RAM
- 38.4 GB
- Headroom
- 11.1 GB
- Generation
- 29.3 tok/sest.
- Prompt processing
- 70.0 tok/sest.
- First token (1K prompt)
- 14.6 sest.
Decode speed is bandwidth ÷ active weight bytes, at 273 GB/s and 78% efficiency. These are modelled figures, not measurements.
Every quantization, on your Mac
| Quant | Weights | + KV | Verdict | Speed est. | Repo |
|---|---|---|---|---|---|
| q4 | 26.3 GB | 27.3 GB | FITS · 11.1 GB FREE | 29.3 tok/s | Mixtral-8x7B-Instruct-v0.1-4bit |
| q8 | 49.6 GB | 50.7 GB | OVER BY 12.3 GB | 15.5 tok/s | Mixtral-8x7B-Instruct-v0.1-hf-4bit-mlx |
We never host weights. Every link goes to Hugging Face.
Run it
Chat
mlx_lm.chat --model mlx-community/Mixtral-8x7B-Instruct-v0.1-4bitServe an OpenAI-compatible endpoint
mlx_lm.server --model mlx-community/Mixtral-8x7B-Instruct-v0.1-4bit --port 8080How far can you push the context
2K tokensFITS · 11.9 GB FREE
cache 0.3 GB
8K tokensFITS · 11.1 GB FREE
cache 1.1 GB
32K tokensFITS · 7.8 GB FREE
cache 4.3 GB
Which Macs run this
| Chip | Smallest RAM that fits | Speed est. |
|---|---|---|
| M1 | no configuration | — |
| M1 Pro | no configuration | — |
| M1 Max | 64 GB | 43.5 tok/s |
| M1 Ultra | 64 GB | 89.3 tok/s |
| M2 | no configuration | — |
| M2 Pro | no configuration | — |
| M2 Max | 64 GB | 44.1 tok/s |
| M2 Ultra | 64 GB | 90.4 tok/s |
| M3 | no configuration | — |
| M3 Pro | no configuration | — |
| M3 Max (14-core CPU) | no configuration | — |
| M3 Max (16-core CPU) | 48 GB | 44.1 tok/s |
| M3 Ultra | 96 GB | 93.7 tok/s |
| M4 | no configuration | — |
| M4 Pro | 48 GB | 29.3 tok/s |
| M4 Max (14-core CPU) | 48 GB | 45.8 tok/s |
| M4 Max (16-core CPU) | 48 GB | 61.7 tok/s |
| M5 | no configuration | — |
| M5 Prounverified | 48 GB | 33.4 tok/s |
| M5 Maxunverified | 48 GB | 70.2 tok/s |