Handbook
Read in order for a full grounding, or jump to the one that is blocking you. Every article is written for someone who owns a Mac, not someone who owns a datacentre.
- 01 · 6 min
Unified memory, explained properly
Why a Mac with 48 GB behaves unlike a PC with a 24 GB graphics card, and where the limit actually sits.
- 02 · 7 min
Quantization without the hand-waving
What 4-bit actually means, why the file is bigger than the arithmetic suggests, and when the quality loss starts to matter.
- 03 · 6 min
The KV cache is the thing that surprises people
Your model fit at 4K context and died at 128K. This is why, and how to calculate it in advance.
- 04 · 6 min
Why tokens per second is a bandwidth problem
Decode speed is almost entirely predictable from memory bandwidth and model size. Here is the arithmetic.
- 05 · 5 min
Mixture-of-experts: big model, small bill at inference
Why a 30B MoE can feel like a 3B model, and what it costs you in memory to get that.
- 06 · 5 min
Context windows: advertised, usable, and affordable
A model that claims 128K rarely gives you 128K useful tokens, and you probably cannot afford them anyway.
- 07 · 4 min
Your first model in about five minutes
Install, download, generate. Nothing to configure, nothing to sign up for.
- 08 · 6 min
How to choose a model without reading a leaderboard
Benchmarks measure benchmarks. Here is a decision procedure based on what you are actually doing.
- 09 · 6 min
Buying a Mac for local AI
Which spec actually changes your experience, and which upgrades are money you will not notice.
- 10 · 5 min
Quantizing a model yourself
When the quant you need does not exist, converting it is one command and about ten minutes.
- 11 · 6 min
Fine-tuning on your own Mac
LoRA on Apple silicon is real and practical. What it can teach a model, and what it cannot.
- 12 · 5 min
What running locally actually protects
The privacy argument for local inference is strong, but it is narrower than the marketing suggests.
- 13 · 6 min
When it does not work
The failures people actually hit on Apple silicon, and what fixes each one.