Tencent's new Hy-Memory-30B-A3B has "memory" in the name, and it doesn't remember a thing between calls. It's a memory writer: a 30B mixture-of-experts model with about 3B active parameters, trained to decide what goes into an external memory store and how old entries get updated. The memory itself still lives in your Mem0, MemOS or Hy-Memory backend, in a vector database you run.
If you only want it running: the Q4_K_M GGUF is 18.2 GB, loads in llama.cpp from release b9993 onward or in Ollama, and there's an MLX build for Apple Silicon. A 24 GB GPU or a 32 GB Mac is the realistic floor. Below are the commands for llama.cpp, Ollama, vLLM and MLX, what the benchmarks actually say (the "beats Claude Opus 5" headline is a 0.87-point margin), and why I think a model that writes memory instead of holding it is the better design.
What "memory" means in Hy-Memory-30B-A3B
The model card says it replaces the general-purpose LLM inside Mem0, MemOS and Hy-Memory. Those frameworks already call an LLM on every write to pull facts out of a conversation and reconcile them with what's already stored ("user moved from Berlin to Seattle" should overwrite, not append). In a lot of setups that call goes to a hosted model like GPT, Claude or DeepSeek, billed per token, on every turn of every user.
Hy-Memory-30B-A3B is a small model trained for exactly that job. Tencent trained it in two stages, both run inside the actual memory harnesses:
- Memory on-policy distillation, where a teacher supervises the student's chain of memory states.
- "Attributed" memory RL, which assigns credit at chunk level and carries the selected memory bank across windows.
The quickstart example on the card shows what it's for. System prompt: extract explicit user facts and preferences, don't infer, return JSON with a memories list of strings. Temperature 0, 512 max tokens, thinking off. That's the whole product: a structured extractor you can host yourself.
The specs
From the card and config.json:
| Spec | Value |
|---|---|
| Architecture |
HYV3ForCausalLM (hy_v3), same family as Tencent's Hy3 |
| Total / active params | 30B / ~3B |
| Experts | 128 routed, 8 per token |
| Layers / hidden size | 48 / 2048 |
| Max position embeddings | 262,144 |
| Languages | English, Chinese |
| License | Apache-2.0 |
| Paper | arXiv 2610.22196 |
Apache-2.0 is the good news here. No custom "community license" with a monthly-active-users clause, so you can put it inside a commercial product without calling legal.
The benchmarks, read carefully
Tencent's headline table swaps only the memory writer and keeps the answer model, judge and retrieval fixed. Averages across six benchmarks (LoCoMo, LongMemEval, PersonaMem-v2, Travel, Shopping, Code Memory Bench):
| Memory writer | Mem0 | Hy-Memory | MemOS | EverMemOS (held out) | Avg |
|---|---|---|---|---|---|
| Hy-Memory-30B-A3B | 54.02 | 60.55 | 50.51 | 56.89 | 55.49 |
| Claude Opus 5 | 53.43 | 60.53 | 50.50 | 54.04 | 54.62 |
| DeepSeek V4.1 Flash | 49.73 | 60.20 | 49.17 | 55.47 | 53.64 |
| Qwen3.6-35B-A3B | 47.51 | 55.87 | 47.36 | 50.75 | 50.37 |
| Qwen3-30B-A3B | 39.81 | 45.67 | 45.44 | 46.57 | 44.37 |
"Beats Claude Opus 5" is technically true and mostly noise. The margin is 0.87 points on average, 0.02 on Hy-Memory, 0.01 on MemOS. These are vendor numbers on a fixed 20% subsample of each benchmark, using Tencent's own harness and judge. OrcaRouter's write-up put it well: the model "has earned the right to be tested, not the right to be believed."
The result I care about is the bottom half of the table. Against the Qwen3-30B-A3B you'd otherwise drop in as a cheap local extractor, it's +11 points on average, and the EverMemOS column is a framework it never saw in training. If you're running Mem0 with a small local model today, that gap is the reason to try it.
The per-benchmark scores are worth a look too. On Shopping, no writer in any harness gets above 24, Opus 5 included, and Hy-Memory itself lands between 18.33 and 21.11, so some memory tasks stay unsolved no matter who writes the memory.
The A3B catch: fast, but all 30B weights still sit in memory
Every A3B model has the same fine print. Only ~3B parameters fire per token, so decode is fast. But the router can pick any of the 128 experts on the next token, so all of them have to be loaded. You pay RAM for 30B and compute for 3B.
Sizes for the published files:
| Build | Size on disk | Source |
|---|---|---|
| BF16 (original) | ~60 GB | model repo |
| GGUF Q8_0 | 32.0 GB | rapatsky GGUF |
| GGUF Q6_K | 24.7 GB | same |
| GGUF Q5_K_M | 21.4 GB | same |
| GGUF Q4_K_M | 18.2 GB | same |
| MLX mixed 4/6-bit | ~17 GB (18.2 GB listed) | hermitdave MLX |
Add KV cache and OS overhead on top. Realistic floor: a 24 GB GPU or a 32 GB Mac for Q4_K_M, and 64 GB if you want Q8_0 with room for a long context. The 262K max context is a config value; the GGUF uploader tested at 32K and says so explicitly.
Run Hy-Memory-30B-A3B with llama.cpp or Ollama
hy_v3 support landed in mainline llama.cpp with PR #25395, shipped in release b9993. Older builds don't know the architecture and won't load the file. Then, using the uploader's own command:
hf download rapatsky/Hy-Memory-30B-A3B-GGUF \
Hy-Memory-30B-A3B-Q5_K_M.gguf chat_template.jinja --local-dir ./hy-memory-gguf
llama-server \
--model ./hy-memory-gguf/Hy-Memory-30B-A3B-Q5_K_M.gguf \
--alias hy-memory --host 127.0.0.1 --port 8081 \
--n-gpu-layers all --flash-attn on \
--ctx-size 32768 --parallel 1 \
--batch-size 512 --ubatch-size 256 \
--cache-type-k f16 --cache-type-v f16 \
--jinja --chat-template-file ./hy-memory-gguf/chat_template.jinja \
--chat-template-kwargs '{"reasoning_effort":"no_think"}'
Ollama works off the same repo: ollama run hf.co/rapatsky/Hy-Memory-30B-A3B-GGUF:Q4_K_M.
Two things from that model card worth knowing before you convert your own GGUF. Tencent's config lists moe_intermediate_size=1536, but the actual expert tensors are 768, and the uploader patched it to 768 during conversion. And their small sanity test (34 handcrafted English dialogues plus 34 Russian ones) got 68/68 valid JSON on every quant, but only 23/34 strict successes in English, also on every quant. The failures were duplicated facts, over-storing and the odd unsupported inference, all wrapped in perfectly valid JSON.
Run it with vLLM (full BF16)
This is the officially documented path. The card says it was tested with vLLM v0.31.0 and expects the served name hy-memory:
pip install vllm
vllm serve tencent/Hy-Memory-30B-A3B --served-model-name hy-memory
That's the BF16 checkpoint, so plan for ~60 GB of weights before KV cache. One 80 GB card, or tensor parallel across two 48 GB cards. Then call it like the card's example:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
model="hy-memory",
messages=[
{"role": "system", "content": "Extract explicit user facts and preferences as concise, "
"self-contained statements. Do not infer. Return JSON: {\"memories\": [...]}"},
{"role": "user", "content": "I'm a software engineer in Seattle, vegetarian, and I prefer Python."},
],
temperature=0.0,
max_tokens=512,
extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
)
print(resp.choices[0].message.content)
Run it with MLX on a Mac
Upstream mlx-lm doesn't know hy_v3 yet. The hermitdave Mixed4-6 build ships its own hy_v3.py and points to it from config.json, so it loads with stock mlx-lm:
pip install -U mlx-lm
mlx_lm.server --model hermitdave/Hy-Memory-30B-A3B-MLX-Mixed4-6 --port 8080
Don't grab the -omlx variants by mistake: they use different expert keys and fail in mlx-lm with "Received 423 parameters not in model." The uploader reports ~65 tok/s decode on an M3 Max with 64 GB, perplexity within 0.94% of the 8-bit build, and 70/70 facts recalled on their extraction test.
Plug it into Mem0 or the Hy-Memory SDK
Tencent's Hy-Memory SDK (pip install hy-memory==2.0.3) accepts any OpenAI-compatible endpoint through MEMORY_LLM_BASE_URL and MEMORY_LLM_API_KEY, so point it at your llama-server or vLLM port. Use pro mode: lite makes no LLM calls at all, which means the writer model never runs. You still need a separate embedding endpoint (MEMORY_EMBEDDER_MODEL, MEMORY_EMBEDDING_DIMS). Mem0 and MemOS take an OpenAI-compatible base URL the same way.
Why I want memory outside the model
I'd rather have memory as a specialist model writing into a store I can inspect than as hidden state baked into weights. When a user says "forget my address," I want to run a DELETE and see the row disappear. Hy-Memory-30B-A3B fits that setup: it makes the expensive per-write LLM call cheap and local, and the memory stays yours.
What it doesn't fix is the write quality ceiling. A 0.87-point lead over Opus 5 on a vendor subsample and a 23/34 strict pass rate on a community test both say the same thing: test it on your own conversations before you trust it with your users' data.
So which side are you on? Do you want memory built into the model, or kept outside it in a vector store you control? And if you're already running Mem0 or similar, what are you using as the extractor today?
Top comments (0)