Skip to content

THE FIELD JOURNAL

How to Install llama.cpp and Run Your First Model

By Local AI Scout · Beginner guide

Paper-cut illustration of a terminal window with colored paper lines, a toolbox with gears and a wrench, and a stack of file blocks feeding a small paper engine

By Local AI Scout · Local · Apps · Guide · Docs-only · Checked November 17, 2026

Short answer: You don't need to compile llama.cpp any more. Its README lists a one-line installer from llama.app, Docker, prebuilt downloads on the Releases page, and package managers: winget install llama.cpp on Windows and brew install llama.cpp on Mac or Linux. Once installed, the README's quick start downloads and runs a small GGUF model from Hugging Face in one command, and a second command starts an OpenAI-compatible server with a built-in chat app called Llama Chat. llama.cpp is free under the MIT license. We copied these commands from the docs and haven't run them for this page.

This guide follows one reader: a hobbyist who has used Ollama for a month and wants to try the engine underneath, mostly to run a GGUF file she downloaded herself.

What llama.cpp is

The project describes itself as "LLM inference in C/C++." Its stated goal is to run large language models "with minimal setup and state-of-the-art performance on a wide range of hardware," locally and in the cloud. It's built on the ggml library and uses the GGUF file format. If you're new to those file names, read What is GGUF? first.

Several friendly apps build on it. LM Studio's docs, for example, describe running "llama.cpp (GGUF) or MLX models." Our Ollama vs llama.cpp vs vLLM comparison covers when each engine makes sense.

Install it (pick one)

OptionCommand or step (from the docs)Good for
Installer, Mac/Linuxcurl -LsSf https://llama.app/install.sh | shQuickest start
Installer, Windows PowerShellirm https://llama.app/install.ps1 | iexQuickest start
Windows package managerwinget install llama.cppEasy updates on Windows
Homebrewbrew install llama.cppMac and Linux
MacPorts or NixListed in docs/install.mdExisting users of those tools
Prebuilt downloadReleases page on GitHubPicking a build for your graphics card
DockerSee the project's Docker docsServers

A note on piping a script into your shell: it runs code from the internet with your permissions. If that makes you uneasy, use winget, Homebrew or a release download instead.

llama.cpp moves fast. The Releases page showed several numbered builds published on November 17, 2026 alone. For the project's milestone release, see our November 14 news roundup.

Run your first model

The README's quick start uses a tiny model, so it downloads fast:

llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

-hf tells llama.cpp to fetch the GGUF file from Hugging Face. That download needs internet; after that, the model runs on your computer. Older tutorials, and parts of the project's own docs, use hyphenated tool names like llama-cli and llama-server. If one form isn't found, try the other; which names you get depends on the build you installed ‡.

Before downloading bigger files, check they'll fit with our memory formula, and read Are AI model downloads safe? if the file comes from someone you don't know.

Start the chat server

llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF

The README says this launches an OpenAI-compatible API server, and that Llama Chat, the built-in chat app, runs against it. Other apps that speak the OpenAI API, like Open WebUI, can connect to it too. Keep it on your own computer unless you know how to secure a server on your network.

Choose llama.cpp if…

  • You want to run a specific GGUF file with full control over settings.
  • You want the newest model support, often before friendlier apps catch up.
  • You're comfortable in a terminal. Some brand-new models need their maker's own llama.cpp fork first; Olala mini is one.

Choose an app on top of it if…

Frequently asked questions

Is llama.cpp free?

Yes. The llama.cpp repository is published under the MIT license, which allows free personal and commercial use. The models you run with it have their own licenses.

Do I need to compile llama.cpp?

No. The README lists an installer script from llama.app, Docker, prebuilt binaries on the Releases page, and package managers such as winget and Homebrew. Building from source is still an option.

Does llama.cpp have a chat interface?

Yes. The README says the server command runs an OpenAI-compatible API with a built-in chat app called Llama Chat. You can also chat in the terminal with the cli command.

Should I use llama.cpp or Ollama?

Use Ollama or LM Studio if you want an easy app. Use llama.cpp if you want to run a specific GGUF file, tune settings directly or get new model support early.

Sources

  • ggml-org/llama.cpp README (https://github.com/ggml-org/llama.cpp), opened November 17, 2026: description, MIT license, quick-start installer, llama cli / llama serve commands, Llama Chat.
  • llama.cpp docs/install.md (https://github.com/ggml-org/llama.cpp/blob/master/docs/install.md), opened November 17, 2026: winget, Homebrew, MacPorts, Nix.
  • llama.cpp Releases (https://github.com/ggml-org/llama.cpp/releases), opened November 17, 2026.
  • LM Studio docs index (https://lmstudio.ai/llms.txt), opened November 17, 2026: "Run llama.cpp (GGUF) or MLX models".

How we checked: a documentation-based guide, not a hands-on test. Sources opened November 17, 2026 unless stated. ‡ marks something we could not confirm on an official page. See our editorial policy.

Keep exploring: Apps · Models · All guides

THE SCOUT LETTER

Good finds.
Less noise.

Useful local AI apps, practical guides and the updates worth your time. A thoughtful note in your inbox.

Free to join. Unsubscribe with one click.