How to choose a model without reading a leaderboard

Start with the constraint, not the leaderboard

Your Mac's memory eliminates most of the catalog before quality is even a question. Set your machine at the top of this page and the list of candidates shrinks to something you can reason about. Only then is it worth comparing models.

Then pick by task

General chat and writing. Instruction-tuned general models in the 7–14B range. Above that the gains are real but modest for conversational use, and the speed penalty is not.

Code. Use a coding-specialised model. A 7B model trained on code will beat a 14B generalist at completion and refactoring, and the specialised ones tend to handle long files better. Set temperature low.

Reasoning and maths. Reasoning models generate long internal chains before answering. They are markedly better on hard problems and markedly slower per useful token, because you pay for the thinking. Worth it for hard problems, irritating for "what is the capital of Peru".

Documents and images. You need a vision-language model. These carry an image encoder on top of the language model, so budget more memory than the parameter count suggests.

Retrieval. A small embedding model plus a mid-size chat model beats one enormous model, at a fraction of the memory.

The size rule of thumb

• Under 4B: fast, fits anywhere, fine for classification, extraction and simple chat. Loses coherence on anything multi-step.
• 7–14B: the sweet spot. Genuinely useful for most work, comfortable on 16–32 GB.
• 27–32B: noticeably better at reasoning and instruction-following. Wants 32–64 GB and patience.
• 70B+: the best quality you can run locally, at speeds that make you re-read your prompt before sending it. 64 GB minimum, 128 GB to be comfortable.

Read the licence before you get attached

Some strong open-weight models carry restrictions on commercial use, on training other models, or on scale of deployment. Every model page here states the licence and whether commercial use is permitted. If this is for work, filter on it first — it is cheaper than discovering the problem after you have built on it.

Test with your own prompts

Public benchmarks are contaminated, gamed, and often measure formats you do not use. Keep five prompts that represent your actual work. Run each candidate against them at temperature 0. Ten minutes of that will tell you more than an afternoon of leaderboard reading.

When in doubt

Take the largest model that shows as "fits" — not "tight" — at your real context length, in a 4-bit quant, from a family with a permissive licence. That single sentence gets most people to a good answer.