DEV Community: Ali Baizhanov

Our Monitoring Said 62% of Retrievals Were Failing. The Bug Was Two Score Scales in One Column.

Ali Baizhanov — Thu, 23 Jul 2026 10:48:13 +0000

The scare

Hybrid retrieval over personal memory — vector similarity + BM25, fused with Reciprocal Rank Fusion, optional cross-encoder rerank on some tiers. Every search logs top_score for quality monitoring. Analyzing 10,706 logged searches, I applied the obvious threshold — top_score < 0.3 = weak retrieval. Result: 62% "failures," a dozen users at "100% failure with avg score 0.017," and a terrifying month-over-month "degradation." One of the "100% failed" users was a paying customer with a thousand searches. I was halfway into incident mode.

The tell

A search for an exact entity name — a guaranteed hit — logged top_score 0.0426. And the "failing" users all averaged 0.016–0.021. Then it clicked: RRF scores are 1/(k + rank) with the standard k=60. Top rank = 1/60 ≈ 0.0167. My "catastrophic" users weren't failing — their top result was rank-1 almost every time. An average of 0.017 is what perfect RRF retrieval looks like.

What actually happened

Requests that go through the reranker log cosine-style scores (0–1 scale, 0.3+ = good). Requests on the raw RRF path log fusion scores (0.016–0.05 scale, where 0.017 = excellent). Both landed in the same top_score column with no scale tag. Every aggregate over that column — means, z-scores, my failure thresholds, even the health-monitoring cron — was averaging apples with orbital velocities. The "month-over-month degradation" was just the RRF-path share growing as more traffic moved to hybrid.

What survived scale-correction: true failure (zero results) was 9–13%, driven mostly by two accounts whose agents were querying literally empty stores — a real problem, but a completely different one than "retrieval is broken."

Lessons that generalize

A fused ranking score is not a similarity. RRF outputs rank information, not confidence. The moment you fuse, the score's absolute value stops meaning what your dashboards think it means.
Never store scores from different scoring regimes in one unlabeled column. Log a score_kind (or a scale-aware quality label computed at write time) — analysis-time guessing is how you get 3am false incidents.
The only scale-free failure signal is emptiness. Zero results means the same thing on every path. When in doubt, count zeros, not thresholds.
Validate your alarm against a known-good query before believing it. One exact-match search that "scored 0.04" saved me from paging myself.

The k=60 default everyone inherits comes from Cormack, Clarke & Buettcher (2009), "Reciprocal Rank Fusion outperforms Condorcet and individual rank learning methods." The trap applies to any RAG stack mixing rerankers with fusion scoring — grep your score column and look for a bimodal cluster around 1/60.

I build Mengram, a memory layer for AI agents. The fix — a scale-aware quality label written alongside every search — is in the public commit history. Questions welcome.

Claude Code Forgets Everything After Compaction. Here's the Fix That Survives It

Ali Baizhanov — Wed, 22 Jul 2026 10:22:49 +0000

The problem: compaction is amnesia by design

When a Claude Code session approaches its context limit, auto-compact summarizes the conversation and throws away the original. It has to — context windows are finite. But what survives is a summary written under token pressure, and what dies is exactly the stuff you needed: the decision you made an hour ago, the constraint you stated once, the approach you already rejected twice.

This isn't a niche complaint. On Anthropic's own issue tracker: enhanced /compact with restorable summaries (114 upvotes), no way to review context after compaction (79), auto-compact erases chat history without warning (35), and — the quiet killer — Claude forgets CLAUDE.md guidance after compaction (28). Hundreds of developers voting on the same wound.

Why CLAUDE.md doesn't save you

The standard advice is "put important context in CLAUDE.md." It helps — until it doesn't. CLAUDE.md is static: it holds what you remembered to write down last week, not the decision from forty minutes ago that compaction just ate. And per the issue above, even CLAUDE.md guidance itself loses force after heavy compaction as the summary crowds it out.

What actually survives: memory outside the context window

The durable fix is structural: keep the important state outside the thing that gets compacted, and re-inject it on every fresh start. Claude Code has the exact machinery for this — the SessionStart hook fires not just on startup, but also on /clear, resume, and after compaction.

That's how the Mengram plugin makes context survive compaction:

During the session, a Stop hook captures each turn in the background — facts, decisions, and workflows get extracted into persistent memory (API keys and tokens are redacted client-side).
After compaction (or /clear, or a new session, or a different machine), the SessionStart hook reloads your cognitive profile — who you are, what you're building, what you decided — as fresh context. The summary can be lossy; the memory isn't.
On every prompt, relevant past context is recalled and injected, so "how did we deploy this again?" gets answered from memory instead of re-derived.

Setup (60 seconds)

# 1. Free API key: https://mengram.io — save it once
mkdir -p ~/.mengram && echo '{"api_key": "om-your-key"}' > ~/.mengram/config.json

# 2. Install the plugin
claude plugin marketplace add alibaizhanov/mengram
claude plugin install mengram@mengram

# 3. Optional: feed in your existing session history (secrets redacted locally)
pip install mengram-ai && mengram import claude-code

Test it: tell Claude something about your project, run /compact (or /clear), and ask again. The context comes back — not from the summary, but from memory.

What this doesn't fix

Honesty section: no external memory restores the full pre-compact transcript — that's gone, and tools claiming otherwise are re-summarizing too. What persistent memory changes is which things survive: instead of whatever the compactor kept under pressure, you keep structured facts, decisions, and workflows extracted while they were fresh. For the transcript itself, vote on #17428 (https://github.com/anthropics/claude-code/issues/17428) — file-backed summaries would compose beautifully with external memory.

I'm the founder of Mengram (https://mengram.io) — a memory layer for AI agents (semantic facts + episodic events + procedural workflows that evolve from failures). Free tier, self-hostable, Apache 2.0. Questions welcome in the comments.

How to Give Claude Code Persistent Memory (2 Commands)

Ali Baizhanov — Fri, 20 Mar 2026 13:07:00 +0000

Claude Code is incredible for coding — but it starts fresh every session. It doesn't know you used Railway yesterday, that you prefer Sora over Inter, or that you spent 3 hours debugging that psycopg2 pool deadlock last week.

Here's how to fix that in 2 commands.

Setup (30 seconds)

pip install mengram-ai
mengram setup

mengram setup creates a free account and installs Claude Code hooks. That's it.

What happens next

Every Claude Code session now does this automatically:

Session Start  →  Loads your cognitive profile (who you are, tech stack, preferences)
Every Prompt   →  Searches past sessions for relevant context
After Response →  Saves new knowledge in background

No manual saves. No tool calls. No MCP configuration. Claude just remembers.

What it actually remembers

Mengram stores 3 types of memory — not just facts:

Semantic memory — facts and preferences:

"Uses Python 3.12, deploys to Railway, prefers PostgreSQL over MySQL"

Episodic memory — events and decisions:

"March 15: Debugged pool deadlock, fixed by increasing pool_max from 2 to 5"

Procedural memory — workflows that evolve:

"Deploy flow: git push → Railway auto-deploys from main → check logs"

The procedural part is key: if a workflow fails, Mengram updates the procedure with what went wrong and how you fixed it. Next time Claude encounters a similar task, it already knows the correct approach.

Quick example

After a few sessions, Claude Code builds a cognitive profile of you:

"Ali is a solo founder building Mengram. Uses Python/Flask on Railway
with Supabase PostgreSQL. Prefers concise code, avoids over-engineering.
Uses gunicorn with 1 worker. Common issues: psycopg2 pool sizing,
OpenAI API rate limits."

When you start a new session and say "fix the deploy", Claude already knows your stack, your deploy process, and what went wrong last time.

How it works under the hood

Mengram uses Claude Code hooks — shell commands that run at specific lifecycle points:

PreToolUse — before each prompt, searches your memory for relevant context
PostToolUse — after each response, extracts and saves new knowledge
SessionStart — loads your cognitive profile into the system prompt

All data goes to Mengram's API where it's chunked, embedded, and stored in a knowledge graph. Search uses hybrid retrieval (vector + graph traversal + reranking).

Beyond Claude Code

Same API works with any AI tool:

from mengram import Mengram
m = Mengram()

# Store a conversation
m.add([
    {"role": "user", "content": "Deploy the app to production"},
    {"role": "assistant", "content": "Pushed to main, Railway auto-deployed. Took 45s."}
])

# Later, in a different session
results = m.search("how do I deploy?")
# → Returns the deploy procedure + past deployment events

Works with MCP (29 tools), LangChain, CrewAI, OpenClaw, and plain REST API.

Pricing

Free tier: 50 adds/month, 300 searches. Enough for personal use. Paid plans start at $5/mo.

Open source, Apache 2.0.

I built an AI memory API with 3 memory types — here's why facts alone aren't enough published

Ali Baizhanov — Wed, 18 Feb 2026 10:08:31 +0000

Every AI memory tool I tried had the same problem: they only store facts.

"User likes Python." "User lives in Almaty." Cool. But human memory doesn't work like that. We have three types:

Semantic — facts and knowledge ("Python is a programming language")
Episodic — events and experiences ("I spent 3 hours debugging that auth bug last Tuesday")
Procedural — how to do things ("Deploy: build → upload → push → verify")

I built Mengram to give AI all three. Here's what I learned building it.

The Problem

I was building AI agents that needed to remember things across sessions. Tried Mem0, tried rolling my own with pgvector. Same issue every time:

My agent could remember that I use Railway for hosting. But it couldn't remember that last Friday's deploy broke because I forgot to run migrations. And it definitely couldn't remember that the correct deploy process is: test → build → push → migrate → verify.

That's three different kinds of memory, and every existing tool only handles the first one.

The Solution: 3 Memory Types from 1 API Call

Mengram extracts all three types automatically:

from mengram.cloud.client import CloudMemory

m = CloudMemory(api_key="om-...")

m.add([
    {"role": "user", "content": "Fixed the auth bug today. The problem was API key cache TTL was set to 0. My debug process: check Railway logs, reproduce locally, fix and deploy."}
])

One call. Mengram's LLM extraction pipeline produces:

Semantic: "API key cache TTL of 0 caused auth bug"
Episodic: "Debugged auth bug, root cause was cache TTL, fixed and deployed"
Procedural: "Debug process: check logs → reproduce locally → fix → deploy"

The Killer Feature: Procedural Learning

This is what no competitor has.

Your AI agent completes a multi-step task. Mengram saves the steps as a procedure with success/failure tracking. Next time a similar task comes up, the agent already knows the optimal path.

Day 1: Agent figures out deployment
  → test → build → push → migrate → verify
  → Mengram saves as procedure (1 success, 0 failures)

Day 5: Agent deploys again  
  → Finds procedure in memory
  → Follows proven path
  → Records success (2 successes, 0 failures)

Day 12: Agent skips tests, deploy breaks
  → Records failure (2 successes, 1 failure)
  → Next time: "This procedure works better with tests first"

The AI literally learns from its own experience. Not from fine-tuning, not from few-shot examples — from actual procedural memory.

Smart Triggers: Memory That Raises Its Hand

Most memory is passive — you ask, it answers. Mengram also has proactive memory:

Reminders: "You mentioned a meeting with Anya tomorrow at 3pm" → fires webhook 1 hour before
Contradictions: Memory says "Anya is vegetarian" → you say "order steaks for dinner with Anya" → alert
Patterns: 3 out of 5 Friday deploys had bugs → "Maybe wait until Monday?"

These fire automatically via webhooks — works with Slack, Discord, OpenClaw, or any endpoint.

Integrations

Mengram works as a memory layer for any AI stack:

Claude Desktop — MCP server, just add to config
LangChain — drop-in MengramMemory class replacing ConversationBufferMemory
CrewAI — 5 tools including mengram_save_workflow for procedural learning
OpenClaw — skill on ClawHub with bash scripts for all channels
Any LLM — REST API + Python/JS SDKs

pip install mengram-ai    # Python
npm install mengram-ai    # JavaScript

Cognitive Profile

One API call generates a system prompt from everything Mengram knows about a user:

profile = m.get_profile()
print(profile["system_prompt"])

You are talking to Ali, a 22-year-old developer in Almaty building Mengram.
He uses Python, PostgreSQL, and Railway. Recently: debugged pgvector deployment,
researched competitors. Workflows: deploys via build→twine→npm→git.
Communicate in Russian/English, direct style, focus on practical next steps.

Insert into any LLM for instant personalization. Replaces your RAG pipeline.

Architecture

Built on PostgreSQL + pgvector. No separate vector database needed.

Your AI Client (Claude, GPT, any LLM)
    │
    ▼
Mengram Cloud API
    ├── LLM Extraction (entities, episodes, procedures)
    ├── Embedding (OpenAI text-embedding-3-large)
    ├── Hybrid Search (vector + full-text + re-ranking)
    ├── Smart Triggers (reminders, contradictions, patterns)
    └── Memory Agents (Curator, Connector, Digest)
    │
    ▼
PostgreSQL + pgvector
    ├── Entities & Facts (semantic)
    ├── Episodes (episodic)
    ├── Procedures (procedural)
    └── Embeddings (1536-dim vectors)

What I Learned

1. Extraction is everything. The quality of your memory system depends entirely on how well you extract structured data from conversations. I went through 3 versions of the extraction prompt before it reliably separated facts from events from procedures.

2. Contradiction detection is harder than it sounds. "I'm vegetarian" and "I love steak" — obvious contradiction. "I prefer dark mode" and "I switched to light mode" — is that a contradiction or an update? LLM-based conflict resolution was the answer.

3. Procedural memory is the moat. Every competitor does semantic memory. Some do episodic. Nobody does procedural with success/failure tracking. This is what makes agents genuinely learn from experience.

Try It

Free tier, no credit card, 60-second setup:

Sign up at mengram.io
pip install mengram-ai
Start adding memories

Open source (Apache 2.0): github.com/AiBaizhanov/mengram

API docs: mengram.io/docs

I'd love feedback — especially from anyone building AI agents. What memory challenges are you running into?