DEV Community

Sh Raj
Sh Raj

Posted on

llama.cpp 1.0 Is Here: Local AI Finally Has a Stable API

On November 13, 2026, the engine behind most of the local AI software you have ever used grew up. llama.cpp — together with ggml, the tensor library underneath it — shipped version 1.0.0, and with it came something the project had never offered before: a promise about the future.

From now on, breaking changes to the public C API will arrive only as major-version bumps. That single sentence, published in the Hugging Face announcement from the ggml-org team, is the real news of this release. The features are nice. The contract is the story.

If you have ever pinned a llama.cpp commit because an update silently renamed a function your app depended on, you know exactly why this matters.

First, what llama.cpp actually is

A quick clarification, because the name confuses people: llama.cpp is not a language model. It is an inference engine written in C and C++, built on top of the ggml tensor library. It is the software that loads a model file — usually in GGUF format — and runs it on your hardware, whether that is a laptop CPU, an Apple Silicon Mac, or a gaming GPU.

A huge share of local AI tooling sits on top of it: desktop chat apps, coding assistants with local models, servers that expose an OpenAI-compatible API from a machine under your desk. The Hugging Face announcement notes that "many downstream applications already depend on our work," and says the engine supports more than 150 architecture variants as well as all mainstream speculative decoding methods. When a project that foundational changes its versioning rules, every app built on it feels it.

flowchart TB
    A[Your app: chat UI, agent, IDE plugin] --> B[llama-server / libllama]
    B --> C[libggml tensor library]
    C --> D[CPU backend]
    C --> E[Metal / Apple Silicon]
    C --> F[CUDA / Vulkan / other GPUs]

Both libraries — libllama and libggml — reached v1.0.0 on the same day, and both now follow semantic versioning.

The promise: semantic versioning with teeth

The versioning rules, as stated in the announcement, are refreshingly specific. They are tied to the public C API exposed through the project's include directory:

Bump What it means
MAJOR A breaking change to the public C API
MINOR A backward-compatible feature, backend update, model-support addition, or API addition
PATCH A bug fix with no API change

"Moving forward, both llama.cpp and ggml will reflect API and ABI incompatibilities with major-version bumps," the Hugging Face post says.

That is a bigger deal than it sounds. Before 1.0, llama.cpp shipped as a rolling project: you pinned a commit, or you took whatever main gave you and fixed what broke. There was no version number that told you whether an update was safe. Now a move from v1.0.0 to v1.0.1 should be a bug-fix update, and a move to v1.1.0 should add things without breaking what you already call. If a future release removes something from the public API, it has to call itself v2.0.0.

One honest caveat, worth stating plainly: the stated rule covers the public C API. It does not freeze model outputs, performance characteristics, every CLI flag, or every HTTP response shape forever. Production users should still read release notes and run their own tests before upgrading. A contract about the API is not a contract about everything.

What 1.0 removes: the breaking cleanup

A stable API contract does not mean v1.0.0 is a drop-in upgrade. The release uses the major-version milestone to clear out deprecated functionality — which is exactly what a 1.0 is for.

From the release notes, the headline removals in llama.h (PR #30565):

  • llama_set_warmup is gone, along with 35 deprecated functions.
  • defrag_thold and the --defrag-thold option are removed. If that flag is sitting in your llama-server launch script, the server will not accept it anymore — remove it before you upgrade.
  • The NextN speculative-decoding API moves from llama-ext.h into the main llama.h header — a promotion, not a removal, but it changes where you include it from.
  • llama_set_embd_state_src is renamed to llama_set_ctx_state_src.
  • clip.h has been removed, with model and subcontext infrastructure now sitting in mtmd.

If you maintain bindings, a plugin, or an app that links against libllama, the first thing to check is PR #30565's removal list. Old bindings will not build against v1.0.0 until they are updated — that is the intended, one-time cost of getting a stable API afterward.

What 1.0 adds

This is still a feature release, not just a freeze. From the project's own release notes:

Memory and performance

  • OCP FP8 KV caches (-ctk/-ctv f8_e4m3), with optional static per-tensor scales. Smaller KV caches mean longer contexts fit in the same VRAM.
  • A host-memory mixture-of-experts expert cache with a configurable CPU share, so MoE models can spill experts to host RAM instead of failing or crawling when VRAM runs out. The release also adds LRFU and ClassLRU replacement strategies for the cache.
  • Loading of external PLE shards.

New models

  • Three new text architectures: Kimi-K3.1-Pro, Ripple (including the Ripple-65B-Preview dense 65B architecture), and xing4_0.
  • drama-3 text-to-speech, and vision support for MageVL-Cowork and Kimi-K3.1.

Server

  • An OpenAI-compatible /v1/audio/speech endpoint, plus a temperature option for /tts — the server keeps growing into a fuller drop-in for OpenAI-shaped clients.
  • Hardened llama-server: fixes covering server-side request forgery (SSRF), cross-origin (CORS) issues, DNS rebinding, and MCP tool cancellation. If you expose llama-server beyond localhost, this is the part of the release to read carefully. Note the honest framing from the release coverage: these are the project's own descriptions of a security and interface cleanup, not a guarantee that every deployment is now safe by default. Treat an exposed inference server like any other network service.
  • A redesigned web UI with chat groups and a markdown editor.

ggml v1.0.0, released the same day, adds FP8 support, configurable YaRN truncation, and fused MoE routing on Apple's Metal for the DeepSeek-V4 and V4.1 family.

Upgrading: a short checklist

If you run llama.cpp today, here is the practical order of operations:

1. Pin the tag for a source build.

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git checkout v1.0.0
Enter fullscreen mode Exit fullscreen mode

2. Search your code for the removed surface. Grep your project — and any vendored bindings — for llama_set_warmup, defrag_thold, and anything you include from llama-ext.h or clip.h. The NextN API now lives in llama.h.

3. Fix your launch scripts. Remove --defrag-thold from any llama-server command line. Then read the rest of the server flags in the release notes, since a few shifted in this release.

4. Test the server endpoints you actually call. The /v1/audio/speech endpoint is new, and the hardening changes may alter behavior for requests that previously slipped through — that is the point of hardening, but verify your client still works, especially if it runs from a browser or a different origin.

5. Try the FP8 KV cache on your longest-context workload. It is the kind of change that either saves you real VRAM or does not matter for your model — measure it on your own prompts rather than trusting a headline number.

Why this release matters beyond llama.cpp

Local AI spent years in a strange position: enormously popular, but built on infrastructure that made no stability promises. Every app that embedded llama.cpp was, in effect, maintaining a fork of its assumptions about the API. That is fine for a hobby project and exhausting for a product.

A 1.0 with semantic versioning changes the incentives. App developers can now plan upgrades instead of dreading them. Downstream projects can declare a dependency on llama.cpp >= 1.0, < 2.0 and mean something by it. And the ggml-org team's rule — that an official llama.cpp release tag is only created when its internal copy of ggml matches the released ggml — keeps system-wide installs through package managers like Debian and Homebrew compatible.

None of this changes what any app does today. Your local chat app will not suddenly get smarter because its engine now has a 1.0 tag. But the next three years of local AI will be built on this foundation, and foundations are exactly where boring stability matters most.

What do you think?

Are you running llama.cpp in anything — a side project, a product, a homelab server? Does the 1.0 stability promise change how you depend on it, or were you already pinning commits and happy with that? I would especially like to hear from anyone who has hit a breaking change in a rolling-release dependency: how do you decide when to upgrade?

Sources


Thanks for reading! I'm Shaswat Raj — I build CampusLoop, IndexFast, SopKit and open-source dev tools.
GitHub: https://github.com/playlikeboy/ · LinkedIn: https://www.linkedin.com/in/sh20raj/ · YouTube: https://youtube.com/@stackshade

Top comments (0)