Gemma 3 27B IT

Google · 27B · up to 128K context · gemma

FITS · 9.7 GB FREE

Fixes Gemma 2's biggest weakness by pushing context out to 128K and adding native image input. On a 32GB Mac at 4-bit it's one of the more well-rounded large models available for both text and pictures.

On a M4 Pro with 48 GB

macOS9.6 GBweights28.7 GBKV cache0.0 GBheadroom9.7 GB

Weights: 28.7 GB
KV cache: 0.0 GBestimated shape
Usable RAM: 38.4 GB
Headroom: 9.7 GB

Generation: 7.4 tok/sest.
Prompt processing: 33.4 tok/sest.
First token (1K prompt): 30.6 sest.

Decode speed is bandwidth ÷ active weight bytes, at 273 GB/s and 78% efficiency. These are modelled figures, not measurements.

Every quantization, on your Mac

Quant	Weights	+ KV	Verdict	Speed est.	Repo
q4	15.2 GB	15.2 GB	FITS · 23.2 GB FREE	14.0 tok/s	gemma-3-27b-it-4bit
q8	28.7 GB	28.7 GB	FITS · 9.7 GB FREE	7.4 tok/s	gemma-3-27b-it-8bit
bf16	54.0 GB	54.0 GB	OVER BY 15.6 GB	3.9 tok/s	gemma-3-27b-it-bf16

We never host weights. Every link goes to Hugging Face.

Run it

Chat

mlx_lm.chat --model mlx-community/gemma-3-27b-it-8bit

Serve an OpenAI-compatible endpoint

mlx_lm.server --model mlx-community/gemma-3-27b-it-8bit --port 8080

How far can you push the context

2K tokensFITS · 9.7 GB FREE

cache 0.0 GB

8K tokensFITS · 9.7 GB FREE

cache 0.0 GB

32K tokensFITS · 9.7 GB FREE

cache 0.0 GB

128K tokensFITS · 9.7 GB FREE

cache 0.0 GB

Which Macs run this

Chip	Smallest RAM that fits	Speed est.
M1	no configuration	—
M1 Pro	no configuration	—
M1 Max	64 GB	11.0 tok/s
M1 Ultra	64 GB	22.6 tok/s
M2	no configuration	—
M2 Pro	no configuration	—
M2 Max	64 GB	11.2 tok/s
M2 Ultra	64 GB	22.9 tok/s
M3	no configuration	—
M3 Pro	no configuration	—
M3 Max (14-core CPU)	no configuration	—
M3 Max (16-core CPU)	48 GB	11.2 tok/s
M3 Ultra	96 GB	23.7 tok/s
M4	no configuration	—
M4 Pro	48 GB	7.4 tok/s
M4 Max (14-core CPU)	48 GB	11.6 tok/s
M4 Max (16-core CPU)	48 GB	15.6 tok/s
M5	no configuration	—
M5 Prounverified	48 GB	8.5 tok/s
M5 Maxunverified	48 GB	17.8 tok/s