I Built an AI That Forgets on Purpose

Adedotun Akindele — Sun, 19 Jul 2026 22:28:43 +0000

Every chat with an AI feels like a fresh start, doesn’t it? It doesn’t remember what you mentioned yesterday, can’t connect notes across sessions, and treats a casual "good morning" the same as a critical lab result. It’s a little frustrating, honestly.

So I built a memory system where information has to earn its place.

Every message gets scored for importance the moment it arrives. If it turns out useful later, its lifespan extends. Once a night, a background job consolidates the valuable stuff into permanent summaries and facts, and lets the rest fade away.

The name Àtúnbí means "reborn" in Yoruba. Raw memory comes and goes. The good stuff sticks around.

I built this for the Qwen Cloud Hackathon, and tested it by tracking two diabetic patients — Ada and Kai — across visits, medication changes, and lab results. But it works for more than healthcare: swap patients for codebases, legal cases, research papers, or your own notes, and the core model stays exactly the same.

The Architecture: Two Paths, One Brain

The system is split into two complementary rhythms to balance speed and depth: one for real-time responses, another for slow, nightly learning. This split is what makes long-term memory feel snappy, not clunky.

Serving Path (Real-Time)

This loop responds in under a second:

Ingest: Normalize every input (voice, PDFs, images, text) into a single text stream
Process: Embed, score, and extract entities from incoming messages
Working Memory: Store raw messages with dynamic expiration timestamps
Retrieval: Query all memory tiers to build context, then generate a response
Feedback loop: Every time a memory is used, its lifespan extends — it earns its spot.

Learning Path (Nightly Dream Phase)

Once every 24 hours, EventBridge triggers a separate job for slow, deep work:

Pulls all Working Memory entries older than 24 hours
Clusters related messages into conversation summaries → saved to Episodic Memory
Extracts standalone facts → saved to Semantic Memory (contradictions mark old facts as superseded, never deleted)
Updates relationships in the Entity + Procedural Memory graph
Prunes low-score, unused entries — this is the intentional forgetting.

The Five Memory Tiers

Tier	Path	Purpose	Lifespan
Working Memory	Serving	Raw messages + embeddings	1h–168h, extended by use
Episodic Memory	Learning	Conversation summaries	Permanent (unless updated)
Semantic Memory	Learning	Verified standalone facts	Permanent, full audit trail
Entity + Procedural	Shared	Relationship graph + learned patterns	Evolves continuously

The Model Router: Why One LLM Isn’t Enough

I used seven specialized Qwen models, split cleanly across the two paths. The biggest lesson: picking the right tool beats using one giant model for everything.

Serving Path Models (fast, frequent use)

qwen3-asr-flash: Audio transcription — handled medical terminology far better than expected
qwen3.5-omni-flash: Vision/image description — great for context, never trust it for exact numbers
text-embedding-v4: 1536-dim vectors for semantic search (8192 token support, rock solid)
qwen-flash: Real-time scorer — rates importance (0–1) and emotional valence (-1 to +1)
qwen-plus-latest: Workhorse — entity extraction, PDF parsing, final response generation
qwen-turbo: Binary relevance filter — keeps only retrieved memories that actually matter

Learning Path Model (slow, daily use)

qwen-max: Heavy summarization and fact extraction for the Dream Phase — accurate, so used sparingly.

The Bug That Hid For Three Days

Most of my debugging time went here: three silent failures, each masking the next.

The demo: A doctor uploads Ada’s blood work PDF showing her HbA1c is 8.2%. The system confidently replies "7.1%" — and doubles down when questioned.

1. Vision Hallucination

I first sent rendered PDF images to the vision model, which invented plausible-sounding but wrong numbers.

Fix: Extract raw text with pymupdf first, then send that text to a text-only model. Text models don’t make up numbers from pixels.

# Before
pages = _render_pdf_pages(file_content)

# After
pdf_text = _extract_pdf_text(file_content)

2. Poisoned Cache

Fixed… but the system still said 7.1%. That first wrong answer had been stored in Working Memory as a "fact," and every query pulled the cached mistake instead of re-reading the file. I had to manually delete it:

DELETE FROM workingmemory WHERE message ILIKE '%HbA1c%7.1%';

(This is exactly why Working Memory expires quickly — bad entries usually flush themselves, but this one got retrieved so often it kept extending its stay!)

3. Silent Truncation

Fixed again… still wrong. Finally, I checked the database: the model output correct values, but my ingestion pipeline cut every chunk at 500 characters. The HbA1c row sat at position 600, so it never made it to disk.

# The culprit
message=f"[{source_label}]: {para[:500]}",

# Now: full documents stored whole; chat messages still chunked
if source_type in ["pdf", "document"]:
    message=f"[{source_label}]: {para}"
else:
    message=f"[{source_label}]: {para[:500]}"

Takeaway: Never trust just the chat response — check the source, model output, and database.

Deployment: Boring Is Beautiful

Everything runs on Alibaba Cloud, kept simple and maintainable:

ECS: Single Docker container (FastAPI + Nginx) for the Serving Path, deployed via GitHub Actions
ApsaraDB RDS: PostgreSQL + pgvector holds all memory tiers
OSS: Stores raw uploads from Ingest
EventBridge: Triggers the nightly Dream Phase cron, which scales to zero when done

One docker run command ties it all together; credentials live in GitHub Secrets. No Kubernetes, no over-engineering — just what works.

Hard-Won Lessons (So You Save Time)

Check your dependencies: pymupdf once missing from requirements.txt crashed my deploy on startup.
Vision describes, text transcribes: Need exact numbers from a document? Extract raw text first.
One size doesn’t fit all: Chat messages and lab reports are different — don’t force the same chunk size on both.
Test background jobs: I polished the real-time pipeline but barely tested the nightly cron. Always verify the parts that run when you’re asleep.

Àtúnbí. Reborn.

Built for the Qwen Cloud Hackathon. View the code on GitHub.

How do you approach long-term memory in LLM apps? What’s the trickiest bug you’ve chased? Let’s talk in the comments!

DEV Community: Adedotun Akindele