Gemma 2 27B IT

Google · 27.2B · up to 8K context · gemma

FITS · 9.5 GB FREE

Wants a 32GB Mac to run 4-bit without squeezing everything else out of memory. It's a strong writer for its size class, but the short 8K context means it's a poor fit for document-heavy work.

On a M4 Pro with 48 GB

macOS9.6 GBweights28.9 GBKV cache0.0 GBheadroom9.5 GB

Weights: 28.9 GB
KV cache: 0.0 GBestimated shape
Usable RAM: 38.4 GB
Headroom: 9.5 GB

Generation: 7.4 tok/sest.
Prompt processing: 33.2 tok/sest.
First token (1K prompt): 30.8 sest.

Decode speed is bandwidth ÷ active weight bytes, at 273 GB/s and 78% efficiency. These are modelled figures, not measurements.

Every quantization, on your Mac

Quant	Weights	+ KV	Verdict	Speed est.	Repo
q4	15.3 GB	15.3 GB	FITS · 23.1 GB FREE	13.9 tok/s	gemma-2-27b-it-4bit
q8	28.9 GB	28.9 GB	FITS · 9.5 GB FREE	7.4 tok/s	gemma-2-27b-it-8bit

We never host weights. Every link goes to Hugging Face.

Run it

Chat

mlx_lm.chat --model mlx-community/gemma-2-27b-it-8bit

Serve an OpenAI-compatible endpoint

mlx_lm.server --model mlx-community/gemma-2-27b-it-8bit --port 8080

How far can you push the context

2K tokensFITS · 9.5 GB FREE

cache 0.0 GB

8K tokensFITS · 9.5 GB FREE

cache 0.0 GB

Which Macs run this

Chip	Smallest RAM that fits	Speed est.
M1	no configuration	—
M1 Pro	no configuration	—
M1 Max	64 GB	10.9 tok/s
M1 Ultra	64 GB	22.4 tok/s
M2	no configuration	—
M2 Pro	no configuration	—
M2 Max	64 GB	11.1 tok/s
M2 Ultra	64 GB	22.7 tok/s
M3	no configuration	—
M3 Pro	no configuration	—
M3 Max (14-core CPU)	no configuration	—
M3 Max (16-core CPU)	48 GB	11.1 tok/s
M3 Ultra	96 GB	23.5 tok/s
M4	no configuration	—
M4 Pro	48 GB	7.4 tok/s
M4 Max (14-core CPU)	48 GB	11.5 tok/s
M4 Max (16-core CPU)	48 GB	15.5 tok/s
M5	no configuration	—
M5 Prounverified	48 GB	8.4 tok/s
M5 Maxunverified	48 GB	17.6 tok/s