What is it?
What it is
LLM inference in C/C++ - Changelog for `libllama` API - Changelog for `llama-server` REST API
Why it exists
LLM inference in C/C++
Who should use it
Teams building with C++ who want an open-source, self-hosted option.
Who should avoid it
Teams that need a fully managed SaaS with enterprise SLAs out of the box.
How it works
A quick walkthrough in plain English
How llama.cpp works
Step 1 of 3
You interact with it
Open llama.cpp, send a request, or connect it to your stack.
Features
Ggml
Advantages
- Open source (MIT)
- Active C++ ecosystem
- Self-hosted deployment options
Disadvantages
- Requires operational ownership for self-hosted setups
- Community support varies by project maturity
Installation
native
# Use a local model file llama-cli -m my_model.gguf # Or download and run a model directly from Hugging Face llama-cli -hf ggml-org/gemma-3-1b-it-GGUF # Launch OpenAI-compatible API server llama-server -hf ggml-org/gemma-3-1b-it-GGUF
FAQ
What is llama.cpp?
llama.cpp is an open-source project licensed under MIT.
What language is llama.cpp built with?
Primary language: C++.
Is it free to use?
Yes. Licensed under MIT. Check the license for commercial use.
Loading documentation…
View on GitHubFeatured in Videos
YouTube tutorials and walkthroughs for llama.cpp
Alternatives
Similar projects ranked by category, topics, and text overlap.
