Ollama
CLI + serverOne command and the model runs on your machine. No limits, no quotas.
localhost:11434/v1
Directory · verified 2 Sep 2026
Run AI models on your own machine
No rate limit to hit, no key to rotate, and nothing leaving the laptop — the only ceiling is your hardware. These are the runtimes worth knowing, from a one-line CLI to the server production actually uses.
Pick by shape rather than by benchmark: Ollama if you want it working in a minute, LM Studio if you would rather not touch a terminal, MLX on an M-series Mac, vLLM when you are serving other people.
LM Studio
Desktop appOllama with a GUI: discover, download and chat locally.
localhost:1234/v1
llama.cpp
EngineThe engine almost everything else runs on. Plain C++, no dependencies.
localhost:8080/v1
Jan
Desktop appOpen source ChatGPT alternative that works 100% offline.
localhost:1337/v1
GPT4All
Desktop appChat with your documents locally, nothing ever leaves the laptop.
Local RAG
vLLM
Inference serverThe serving engine production runs on. PagedAttention and continuous batching, on your own GPU.
localhost:8000/v1
MLX
Apple SiliconApple’s array framework. On an M-series Mac it is the fastest way to run a model locally.
localhost:8080/v1
llamafile
Single binaryOne file that is both the model and the runtime. Download, chmod +x, run. No install at all.
localhost:8080/v1
KoboldCpp
Single binaryA single executable around llama.cpp, tuned for long-form and creative writing.
GGUF
Questions about these
How much RAM do I need?
A 7B model quantised to 4-bit fits in about 5 GB, so 8 GB of RAM runs one comfortably. 70B models want 40 GB or more. Can I Run AI will score your specific machine.
Is a local model as good as a free API?
Not usually. The models you can run at home are smaller than the ones NVIDIA or Groq serve for free. Local wins on privacy, offline use and having no limits at all.