MLC LLM
A compiler-first runtime that targets Metal, CUDA and mobile from one graph.
When to use it
Use this when one model definition also has to run well on Android and iOS via TVM compilation, not just on your Mac.
Alternatives in Runtimes
MLX
Apple's own array framework, built so unified memory stops being a compromise.
MLX LM
The package behind every `mlx_lm.generate` command on this site.
MLX Swift
MLX's tensor core, ported so Swift apps don't need a Python bridge.
llama.cpp
The C++ runtime that made GGUF and Metal inference normal.
Candle
Hugging Face's minimalist Rust tensor library, Metal backend included.