By Local AI Scout · Local · Apps · Guide · Docs-only · Checked November 17, 2026
Short answer: You don't need to compile llama.cpp any more. Its README lists a one-line installer from llama.app, Docker, prebuilt downloads on the Releases page, and package managers: winget install llama.cpp on Windows and brew install llama.cpp on Mac or Linux. Once installed, the README's quick start downloads and runs a small GGUF model from Hugging Face in one command, and a second command starts an OpenAI-compatible server with a built-in chat app called Llama Chat. llama.cpp is free under the MIT license. We copied these commands from the docs and haven't run them for this page.
This guide follows one reader: a hobbyist who has used Ollama for a month and wants to try the engine underneath, mostly to run a GGUF file she downloaded herself.
What llama.cpp is
The project describes itself as "LLM inference in C/C++." Its stated goal is to run large language models "with minimal setup and state-of-the-art performance on a wide range of hardware," locally and in the cloud. It's built on the ggml library and uses the GGUF file format. If you're new to those file names, read What is GGUF? first.
Several friendly apps build on it. LM Studio's docs, for example, describe running "llama.cpp (GGUF) or MLX models." Our Ollama vs llama.cpp vs vLLM comparison covers when each engine makes sense.
Install it (pick one)
| Option | Command or step (from the docs) | Good for |
|---|---|---|
| Installer, Mac/Linux | curl -LsSf https://llama.app/install.sh | sh | Quickest start |
| Installer, Windows PowerShell | irm https://llama.app/install.ps1 | iex | Quickest start |
| Windows package manager | winget install llama.cpp | Easy updates on Windows |
| Homebrew | brew install llama.cpp | Mac and Linux |
| MacPorts or Nix | Listed in docs/install.md | Existing users of those tools |
| Prebuilt download | Releases page on GitHub | Picking a build for your graphics card |
| Docker | See the project's Docker docs | Servers |
A note on piping a script into your shell: it runs code from the internet with your permissions. If that makes you uneasy, use winget, Homebrew or a release download instead.
llama.cpp moves fast. The Releases page showed several numbered builds published on November 17, 2026 alone. For the project's milestone release, see our November 14 news roundup.
Run your first model
The README's quick start uses a tiny model, so it downloads fast:
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF
-hf tells llama.cpp to fetch the GGUF file from Hugging Face. That download needs internet; after that, the model runs on your computer. Older tutorials, and parts of the project's own docs, use hyphenated tool names like llama-cli and llama-server. If one form isn't found, try the other; which names you get depends on the build you installed ‡.
Before downloading bigger files, check they'll fit with our memory formula, and read Are AI model downloads safe? if the file comes from someone you don't know.
Start the chat server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
The README says this launches an OpenAI-compatible API server, and that Llama Chat, the built-in chat app, runs against it. Other apps that speak the OpenAI API, like Open WebUI, can connect to it too. Keep it on your own computer unless you know how to secure a server on your network.
Choose llama.cpp if…
- You want to run a specific GGUF file with full control over settings.
- You want the newest model support, often before friendlier apps catch up.
- You're comfortable in a terminal. Some brand-new models need their maker's own llama.cpp fork first; Olala mini is one.
Choose an app on top of it if…
- You'd rather click than type: LM Studio or Ollama. Our free local AI apps compared page lists licenses and costs.
Frequently asked questions
Is llama.cpp free?
Yes. The llama.cpp repository is published under the MIT license, which allows free personal and commercial use. The models you run with it have their own licenses.
Do I need to compile llama.cpp?
No. The README lists an installer script from llama.app, Docker, prebuilt binaries on the Releases page, and package managers such as winget and Homebrew. Building from source is still an option.
Does llama.cpp have a chat interface?
Yes. The README says the server command runs an OpenAI-compatible API with a built-in chat app called Llama Chat. You can also chat in the terminal with the cli command.
Should I use llama.cpp or Ollama?
Use Ollama or LM Studio if you want an easy app. Use llama.cpp if you want to run a specific GGUF file, tune settings directly or get new model support early.
Sources
- ggml-org/llama.cpp README (https://github.com/ggml-org/llama.cpp), opened November 17, 2026: description, MIT license, quick-start installer, llama cli / llama serve commands, Llama Chat.
- llama.cpp docs/install.md (https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md), opened November 17, 2026: winget, Homebrew, MacPorts, Nix.
- llama.cpp Releases (https://github.com/ggml-org/llama.cpp/releases), opened November 17, 2026.
- LM Studio docs index (https://lmstudio.ai/llms.txt), opened November 17, 2026: "Run llama.cpp (GGUF) or MLX models".
How we checked: a documentation-based guide, not a hands-on test. Sources opened November 17, 2026 unless stated. ‡ marks something we could not confirm on an official page. See our editorial policy.
