What running locally actually protects

What is genuinely true

When a model runs on your Mac, your prompt does not leave the machine. There is no request to a provider, no server-side log, no retention policy to read, no possibility of your text being used for training, and no third party that can be subpoenaed for it. Pull the network cable and it still works.

For anyone handling client material, health information, unpublished work, or source code under NDA, that is not a marginal benefit. It removes a category of risk rather than mitigating it.

What it does not do

Downloading is not private. Fetching weights from Hugging Face is an ordinary HTTP request that reveals which model you want. Download once and you are done, but the download itself is visible.

The model is not audited. Weights are opaque binaries from whoever uploaded them. They can carry biases, backdoored behaviours, and in the worst case, in some formats, executable payloads. Prefer well-known repositories and organisations with a track record. Safetensors is safer than pickle formats by design.

Your wrapper may phone home. The model is local; the app around it might not be. Check whether your GUI sends telemetry, syncs conversation history, or falls back to a cloud API when a local model is unavailable. That fallback is where people leak data without realising.

Retrieval touches your files. A RAG setup reads your documents and writes an index. That index is as sensitive as the source material and is often left unencrypted in a cache directory.

Local does not mean secure. Anyone with access to your Mac has access to your conversation history and any index you have built. Disk encryption and normal machine hygiene still apply.

A practical checklist

• Prefer safetensors over pickle formats.
• Download from organisations you recognise, and check the repository is what it claims.
• Audit your GUI's network activity once, with Little Snitch or equivalent, before you trust it with anything sensitive.
• Turn off any cloud fallback explicitly rather than assuming it is off.
• Keep RAG indexes on an encrypted volume.
• Remember conversation logs are plain files on disk unless you delete them.

The honest summary

Local inference eliminates the largest privacy risk — sending your data to someone else's computer — and leaves you responsible for the rest. That is a good trade, and it is a better one when you know exactly where the remaining edges are.