Qwen3 Embedding 4B
Alibaba · 4B · up to 32K context · apache-2.0
Built on the Qwen3 4B backbone, this trades embedding model minimalism for MTEB-leaderboard-level retrieval quality, at the cost of needing a 16GB Mac rather than fitting anywhere. It also supports instruction-aware embeddings, so you can bias retrieval toward a task description at query time.
On a M4 Pro with 48 GB
macOS9.6 GBweights4.3 GBKV cache1.2 GBheadroom32.9 GB
- Weights
- 4.3 GB
- KV cache
- 1.2 GB
- Usable RAM
- 38.4 GB
- Headroom
- 32.9 GB
- Generation
- 50.1 tok/sest.
- Prompt processing
- 226 tok/sest.
- First token (1K prompt)
- 4.5 sest.
Decode speed is bandwidth ÷ active weight bytes, at 273 GB/s and 78% efficiency. These are modelled figures, not measurements.
Every quantization, on your Mac
| Quant | Weights | + KV | Verdict | Speed est. | Repo |
|---|---|---|---|---|---|
| q4 | 2.3 GB | 3.5 GB | FITS · 34.9 GB FREE | 94.6 tok/s | Qwen3-Embedding-4B-4bit-DWQ |
| q8 | 4.3 GB | 5.5 GB | FITS · 32.9 GB FREE | 50.1 tok/s | Qwen3-Embedding-4B-mxfp8 |
We never host weights. Every link goes to Hugging Face.
Run it
Chat
mlx_lm.chat --model mlx-community/Qwen3-Embedding-4B-mxfp8Serve an OpenAI-compatible endpoint
mlx_lm.server --model mlx-community/Qwen3-Embedding-4B-mxfp8 --port 8080How far can you push the context
2K tokensFITS · 33.8 GB FREE
cache 0.3 GB
8K tokensFITS · 32.9 GB FREE
cache 1.2 GB
32K tokensFITS · 29.3 GB FREE
cache 4.8 GB
Which Macs run this
| Chip | Smallest RAM that fits | Speed est. |
|---|---|---|
| M1 | 8 GB | 11.5 tok/s |
| M1 Pro | 16 GB | 35.8 tok/s |
| M1 Max | 32 GB | 74.4 tok/s |
| M1 Ultra | 64 GB | 152 tok/s |
| M2 | 8 GB | 17.2 tok/s |
| M2 Pro | 16 GB | 36.2 tok/s |
| M2 Max | 32 GB | 75.3 tok/s |
| M2 Ultra | 64 GB | 154 tok/s |
| M3 | 8 GB | 17.2 tok/s |
| M3 Pro | 18 GB | 26.5 tok/s |
| M3 Max (14-core CPU) | 36 GB | 55.1 tok/s |
| M3 Max (16-core CPU) | 48 GB | 75.3 tok/s |
| M3 Ultra | 96 GB | 160 tok/s |
| M4 | 16 GB | 20.9 tok/s |
| M4 Pro | 24 GB | 50.1 tok/s |
| M4 Max (14-core CPU) | 36 GB | 78.1 tok/s |
| M4 Max (16-core CPU) | 48 GB | 105 tok/s |
| M5 | 16 GB | 27.0 tok/s |
| M5 Prounverified | 24 GB | 57.1 tok/s |
| M5 Maxunverified | 36 GB | 120 tok/s |