Gemma 2 9B IT
Google · 9.2B · up to 8K context · gemma
A capable writer for a 16GB Mac, but the 8K context ceiling is dated next to Qwen3's 40K. Google's safety tuning also makes it noticeably more cautious in refusals than comparable models.
On a M4 Pro with 48 GB
macOS9.6 GBweights9.8 GBKV cache0.0 GBheadroom28.6 GB
- Weights
- 9.8 GB
- KV cache
- 0.0 GBestimated shape
- Usable RAM
- 38.4 GB
- Headroom
- 28.6 GB
- Generation
- 21.7 tok/sest.
- Prompt processing
- 97.7 tok/sest.
- First token (1K prompt)
- 10.5 sest.
Decode speed is bandwidth ÷ active weight bytes, at 273 GB/s and 78% efficiency. These are modelled figures, not measurements.
Every quantization, on your Mac
| Quant | Weights | + KV | Verdict | Speed est. | Repo |
|---|---|---|---|---|---|
| q4 | 5.2 GB | 5.2 GB | FITS · 33.2 GB FREE | 41.0 tok/s | gemma-2-9b-it-4bit |
| q8 | 9.8 GB | 9.8 GB | FITS · 28.6 GB FREE | 21.7 tok/s | gemma-2-9b-it-8bit |
We never host weights. Every link goes to Hugging Face.
Run it
Chat
mlx_lm.chat --model mlx-community/gemma-2-9b-it-8bitServe an OpenAI-compatible endpoint
mlx_lm.server --model mlx-community/gemma-2-9b-it-8bit --port 8080How far can you push the context
2K tokensFITS · 28.6 GB FREE
cache 0.0 GB
8K tokensFITS · 28.6 GB FREE
cache 0.0 GB
Which Macs run this
| Chip | Smallest RAM that fits | Speed est. |
|---|---|---|
| M1 | 16 GB | 5.0 tok/s |
| M1 Pro | 16 GB | 15.5 tok/s |
| M1 Max | 32 GB | 32.2 tok/s |
| M1 Ultra | 64 GB | 66.0 tok/s |
| M2 | 16 GB | 7.4 tok/s |
| M2 Pro | 16 GB | 15.7 tok/s |
| M2 Max | 32 GB | 32.6 tok/s |
| M2 Ultra | 64 GB | 66.8 tok/s |
| M3 | 16 GB | 7.4 tok/s |
| M3 Pro | 18 GB | 11.5 tok/s |
| M3 Max (14-core CPU) | 36 GB | 23.8 tok/s |
| M3 Max (16-core CPU) | 48 GB | 32.6 tok/s |
| M3 Ultra | 96 GB | 69.2 tok/s |
| M4 | 16 GB | 9.0 tok/s |
| M4 Pro | 24 GB | 21.7 tok/s |
| M4 Max (14-core CPU) | 36 GB | 33.8 tok/s |
| M4 Max (16-core CPU) | 48 GB | 45.6 tok/s |
| M5 | 16 GB | 11.7 tok/s |
| M5 Prounverified | 24 GB | 24.7 tok/s |
| M5 Maxunverified | 36 GB | 51.9 tok/s |