Skip to content
mlx.app

Your first model in about five minutes

4 minute read

Install, download, generate. Nothing to configure, nothing to sign up for.

What you need

A Mac with Apple silicon (M1 or later) and macOS 13.5 or newer. Python 3.9 or later. That is the entire list. No account, no API key, no GPU rental.

Install

pip install mlx-lm

Use a virtual environment if you like keeping your system Python clean:

python3 -m venv ~/.venvs/mlx && source ~/.venvs/mlx/bin/activate pip install mlx-lm

Generate

mlx_lm.generate --model mlx-community/Llama-3.2-3B-Instruct-4bit \ --prompt "Explain unified memory in two sentences."

The first run downloads the weights to ~/.cache/huggingface. After that it is instant. A 3B model at 4-bit is around 1.8 GB, so this works on any Apple silicon Mac including an 8 GB base model.

Have a conversation

mlx_lm.chat --model mlx-community/Llama-3.2-3B-Instruct-4bit

This keeps the KV cache between turns, so follow-up questions are fast and the model remembers what you said.

Serve it to your other apps

mlx_lm.server --model mlx-community/Llama-3.2-3B-Instruct-4bit --port 8080

That is an OpenAI-compatible endpoint at http://localhost:8080/v1. Point any client that accepts a custom base URL at it — including the OpenAI Python SDK, with any string as the API key.

Useful flags

  • --max-tokens 1000 — let it write more before stopping.
  • --temp 0.0 — deterministic output, good for code and extraction.
  • --kv-bits 8 — halve the cache cost on long conversations.
  • --max-kv-size 8192 — cap memory growth in long chats.

Then what

Use the fit calculator on this site with your actual Mac selected, and move up in size until a model shows as tight rather than fitting comfortably. That is the largest model you can run well. Somewhere between 7B and 14B at 4-bit is where most people settle: good enough to be useful, fast enough to be pleasant.

If you would rather not use a terminal

LM Studio gives you the same MLX engine behind a GUI, with model browsing and one-click downloads. It is the right recommendation for anyone who does not want to think about Python. The tools directory on this site lists the alternatives with an honest note on when each one is the right choice.