When it does not work

Everything slowed to a crawl

Almost always swapping. The model plus its KV cache exceeded usable memory, so macOS started paging weights to SSD, and decode reads every weight per token. Check memory pressure in Activity Monitor — if it is red, this is your problem.

Fix by shrinking something: a lower quantization, a smaller model, a shorter context, or an 8-bit KV cache. It is not a tuning problem; you are over budget.

Out of memory, but the model should fit

The GPU has its own wired-memory limit, typically well below your installed RAM. Raise it:

Set it to installed RAM minus at least 8 GB. It resets on reboot. Also close other memory-hungry applications first — a browser with many tabs can easily hold several gigabytes.

It crashes only on long conversations

The KV cache grows with every token. Cap it:

The download keeps failing or restarting

Large repositories over an unstable connection. Resume rather than restart — the Hugging Face cache is incremental, so re-running the same command continues where it left off. If the cache is corrupted, delete the specific model directory under ~/.cache/huggingface/hub and retry.

Model not supported / unknown architecture

The architecture is newer than your mlx-lm. Upgrade first:

If it still fails, the architecture has not been implemented yet. Check the mlx-lm repository issues before assuming your file is broken.

Output is repetitive or degenerate

Usually sampler settings, not the model. Temperature 0 makes repetition more likely on long generations; try 0.7 with a repetition penalty. If output is genuinely incoherent rather than repetitive, suspect the quantization — 3-bit conversions of small models frequently produce this.

The chat template is wrong

Symptoms: the model talks to itself, emits role markers, or ignores instructions. Use mlx_lm.chat or the server endpoint, both of which apply the tokenizer's chat template. Hand-writing prompts for an instruction-tuned model is the most common cause of "this model is bad".

It is slower than the estimate on this site

Check, in order: are you on battery (throttling), is another process using the GPU, is memory pressure elevated, and is your prompt long enough that you are measuring prefill rather than decode. Our estimates assume a plugged-in, unloaded machine. If it is still far off, submit the measurement — mismatches are the most useful data we get.

Nothing here matches

Post the exact command, the model repo, your chip and RAM, your mlx-lm version, and the full error. That set of six facts resolves most questions immediately; anything less usually means a round trip.