Mistral Small 24B Instruct 2501

Mistral AI · 24B · up to 32K context · apache-2.0

FITS · 11.6 GB FREE

Aimed squarely at the 32GB Mac tier and it earns the memory, with quality that tracks close to models twice its size on everyday tasks. It's fully Apache-licensed, which matters if you're shipping a commercial product on top of it.

On a M4 Pro with 48 GB

macOS9.6 GBweights25.5 GBKV cache1.3 GBheadroom11.6 GB

Weights: 25.5 GB
KV cache: 1.3 GB
Usable RAM: 38.4 GB
Headroom: 11.6 GB

Generation: 8.4 tok/sest.
Prompt processing: 37.6 tok/sest.
First token (1K prompt): 27.2 sest.

Decode speed is bandwidth ÷ active weight bytes, at 273 GB/s and 78% efficiency. These are modelled figures, not measurements.

Every quantization, on your Mac

Quant	Weights	+ KV	Verdict	Speed est.	Repo
q4	13.5 GB	14.8 GB	FITS · 23.6 GB FREE	15.8 tok/s	Mistral-Small-24B-Instruct-2501-4bit
q8	25.5 GB	26.8 GB	FITS · 11.6 GB FREE	8.4 tok/s	Mistral-Small-24B-Instruct-2501-8bit

We never host weights. Every link goes to Hugging Face.

Run it

Chat

mlx_lm.chat --model mlx-community/Mistral-Small-24B-Instruct-2501-8bit

Serve an OpenAI-compatible endpoint

mlx_lm.server --model mlx-community/Mistral-Small-24B-Instruct-2501-8bit --port 8080

How far can you push the context

2K tokensFITS · 12.6 GB FREE

cache 0.3 GB

8K tokensFITS · 11.6 GB FREE

cache 1.3 GB

32K tokensFITS · 7.5 GB FREE

cache 5.4 GB

Which Macs run this

Chip	Smallest RAM that fits	Speed est.
M1	no configuration	—
M1 Pro	no configuration	—
M1 Max	64 GB	12.4 tok/s
M1 Ultra	64 GB	25.4 tok/s
M2	no configuration	—
M2 Pro	no configuration	—
M2 Max	64 GB	12.5 tok/s
M2 Ultra	64 GB	25.7 tok/s
M3	no configuration	—
M3 Pro	no configuration	—
M3 Max (14-core CPU)	no configuration	—
M3 Max (16-core CPU)	48 GB	12.5 tok/s
M3 Ultra	96 GB	26.7 tok/s
M4	no configuration	—
M4 Pro	48 GB	8.4 tok/s
M4 Max (14-core CPU)	48 GB	13.0 tok/s
M4 Max (16-core CPU)	48 GB	17.6 tok/s
M5	no configuration	—
M5 Prounverified	48 GB	9.5 tok/s
M5 Maxunverified	48 GB	20.0 tok/s