Run Sage-A1.5 as a research agent on your own card and the surprises are not in the model. They hide in a reasoning flag, a request limit, a token budget and one system prompt string.
Sage-A1.5 comes from Shanghai AI Laboratory (the InternScience organisation on Hugging Face), with a paper by Wenjie Lou and 98 co-authors. It is retrained from the Qwen team's open 35B model and the lab's earlier Agents-A1. The authors publish an official Q4_K_M GGUF, a compressed single-file version of about 22 GB, under Apache-2.0. Stock llama.cpp, the open server that runs models on one machine, loads it without changes. We ran it on one RTX 4090 on our cloud; every measurement is in the Sage-A1.5 single-GPU research write-up.
Why it fits on 24 GB, and what that means for sizing
It is a mixture-of-experts model: the weights are split into 256 small specialist blocks, and each token (roughly a word piece) goes through 8 of them plus one shared block. So of 35 billion parameters on disk, about 3 billion do work per token. Speed follows the 3 billion. Memory follows the 35.
Size the card for the whole file plus context, the conversation the model keeps in memory. With 32,768 tokens of context and two parallel slots (requests served at once), text only, it peaked at 21.3 of the card's 23.5 GiB. The card's 64K setup with vision peaked at 23.1 GiB, 128K text only at 23.4 GiB, with no loss of speed. At 32K it wrote about 136 tokens per second for one request, first token after about a quarter second.
Below 24 GB, llama.cpp can park experts in system RAM (--n-cpu-moe). We only simulated smaller cards: a simulated 16 GB card ran about 5.6 times slower and still got the research question right. We never tried real 16 and 12 GB cards, the RTX 3090 or the uncompressed model.
Trap one: the model card's reasoning flag
The llama.cpp command in the model card will not start.
--reasoning-format deepthink # llama-server exits: "Unknown reasoning format: deepthink"
--reasoning-format deepseek # what llama.cpp accepts
The flag tells the server how to split thinking from answer. Even a much newer llama.cpp lacks deepthink. Our example uses deepseek; the authors' card still carries the old value.
Trap two: sub-agents that wait in line
Agentic Orchestration gives the lead model one tool that hands out tasks to sub-agents, copies of the model that search on their own and report back. Asked for the years between the discoveries of hafnium, rhenium and technetium, the lead spawned three sub-agents at once, right in all three test runs. Every tool call arrived in the exact format our code needed.
"At once" needs a server that takes several requests. Our platform's default after a deploy is one request in flight, so the sub-agents queued and the run took 72.0 seconds. With the limit at two, runs took 50 and 56 seconds. That default has not changed yet. Serving it yourself? Check your slot count (-np) and any proxy in front.
Trap three: the thinking eats your token budget
The model reasons through hidden tokens before it answers, and those count against max_tokens. A two-sentence definition spent 1,905 tokens reasoning and took 14.4 seconds.
With max_tokens at 4,096, our first plain question returned 4,096 tokens of "Thinking Process:" and no answer. Our non-streaming endpoint also dropped the reasoning field once content was set. The example asks for 16,384 and streams the agent loops, getting tokens as they are written. Plan for research-job latency, not chat-box latency.
Verification Mode: measure it, do not assume it
The authors' Verification Mode tells the model to check its work before answering. On BrowseComp, a benchmark of hard-to-find web facts, they report it lifting the score from 72.5% to 82.9% over all 1,266 questions. We did not reproduce their benchmarks.
On our easy, Wikipedia-backed questions it changed no answer and doubled the work:
| Off | On | |
|---|---|---|
| Answers right | 6 of 6 | 6 of 6 |
| Average time | 11.2 s | 25.0 s |
| Average tokens written | 934 | 1,890 |
The added turns re-read facts the model already had. Runs varied: in the second hafnium pair, the checked run used one tool call less than the unchecked one.
The switch lives in the chat template, the script that turns messages into the model's prompt. Directly against llama-server, the documented chat_template_kwargs={"verification": true} works, with a non-empty system message. Our endpoint does not pass those kwargs yet. So the example appends the template's trigger line to the system prompt, with a lowercase "protocol" (the project page capitalises it, the template does not). We checked that the model sees the same prompt both ways.
Log wall time, tool calls, completion tokens and correctness per question, off and on. Try it on your hard cases and keep it off for lookups.
What this does not prove
A handful of easy questions, one agent at a time, on one card, exercises tool calls, task hand-off and answers. It says nothing about research quality at scale.
The GPU memory ladder, speed table, deploy timings and every issue with its status are in the complete Sage-A1.5 on one RTX 4090 test. Runnable code: the Sage-A1.5 SDK example.
Credit to the Sage-A1.5 team for the model, both modes and the GGUF, to the Qwen team for the base model, to the llama.cpp project for the server, to Wikipedia for its public API, and to AI Search, whose news video pointed us to the model. Before your first deploy, read the problems and fixes section of the Sage-A1.5 article.
Top comments (0)