Fine-tuning on your own Mac

6 minute read

LoRA on Apple silicon is real and practical. What it can teach a model, and what it cannot.

What fine-tuning is for

Fine-tuning teaches a model a style, a format, or a domain vocabulary. It is very good at "always answer in this JSON shape", "write in our house voice", "use our internal terminology correctly".

It is bad at teaching facts. If you want the model to know your documents, use retrieval — embed the documents, search them, and put the relevant passages in the prompt. Retrieval is cheaper, updates instantly, and can cite sources. Fine-tuning facts into weights tends to produce confident nonsense at the edges.

LoRA in one paragraph

Full fine-tuning updates every weight and needs far more memory than inference. LoRA freezes the original weights and trains two small matrices per layer whose product is added to the original. You train perhaps 0.1 percent of the parameters, the adapter file is a few megabytes, and it runs comfortably on a Mac.

Data format

A JSONL file, one example per line:

{"text": "<s>[INST] Summarise this ticket [/INST] ...</s>"}

Or, for chat models, the messages format:

{"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

Put train.jsonl and valid.jsonl in a folder. A few hundred high-quality, consistent examples beat several thousand sloppy ones — consistency is what the model learns.

Train

mlx_lm.lora --model mlx-community/Qwen3-8B-4bit \
  --train --data ./data \
  --iters 600 --batch-size 4 --lora-layers 16

QLoRA — training an adapter on top of an already-quantized base — is what makes this fit on a laptop. Expect roughly the model's inference footprint plus a few gigabytes.

Test, fuse, run

mlx_lm.generate --model mlx-community/Qwen3-8B-4bit --adapter-path ./adapters --prompt "..."
mlx_lm.fuse --model mlx-community/Qwen3-8B-4bit --adapter-path ./adapters --save-path ./my-model

Keeping the adapter separate lets you swap behaviours against one base model. Fusing gives you a single self-contained model to distribute.

Reading the loss

Watch validation loss, not training loss. When validation stops improving while training loss keeps dropping, you are memorising. Stop, or lower the learning rate. Overfitting on a small dataset happens quickly and shows up as a model that repeats your examples verbatim.

Realistic expectations

A few hundred examples and an hour on an M-series Mac will reliably change tone and output format. It will not add a new capability the base model lacked. Choose the strongest base model that fits, then teach it your shape.