Migrating AI Agent Memory to an Open Format: A Qdrant Adapter Story

AI agents are getting memory, and that's great — until you want to switch memory backends. That's the problem I spent a week on: migrating vector-store-backed agent memory into an open, portable format without losing a single record.

The Problem: Locked-In Agent Memory

Most agent memory systems (Mem0, LangChain, LlamaIndex) persist embeddings and payloads in a vector database like Qdrant, Pinecone, Weaviate, or Redis. The data lives in a proprietary collection schema, and exporting it in a lossless, portable way is a pain nobody really solves.

Then I found Memanto — a project that defines an open knowledge format (OKF) for agent memory. Their bounty asked for migration adapters: take a memory store from a popular vector DB, export it to the OKF bundle, and prove the round trip is lossless.

I took the Qdrant path — the most common production vector store backing agent memory today (Mem0's default, plus LangChain and LlamaIndex integrations).

The Adapter Architecture

The adapter is a pipeline with four stages:

Qdrant collection
    │  (1) exporter — pull all points + payloads
    ▼
records.jsonl
    │  (2) mapper — Qdrant schema → OKF schema
    ▼
OKF bundle (manifest + records)
    │  (3) round-trip validator — re-ingest and diff
    ▼
parity report (61/61 records)

Stage 1 — Exporter. Connect to the Qdrant collection, page through all points (scroll with limit=100 and the offset cursor), and dump each point's id, vector, and payload to a JSONL file. The cursor pagination matters — Qdrant's scroll API returns an next_page_offset that you must follow until it's null. Getting this wrong silently truncates collections.

Stage 2 — Mapper. This is where the real work lives. Qdrant points have id (UUID or unsigned int), vector, and payload (arbitrary JSON). OKF wants a normalized record structure. The mapper handles:

• ID normalization (Qdrant accepts both UUID and numeric point IDs — the mapper canonicalizes them)
• Vector extraction and float32 precision preservation
• Payload field mapping with a configurable field map
• Nested payload objects (payloads are often { "metadata": {...}, "content": "...", "embedding": {...} })

Stage 3 — Round-trip validation. The part that separates a real adapter from a demo: re-import the OKF bundle into a fresh Qdrant collection and diff every record against the source. My validator checks:

• Record count (source vs restored)
• Per-record ID match
• Per-record payload JSON equality
• Per-record vector equality (with an epsilon for float drift)

The Parity Report

The end-to-end run over a real source collection produced:

Records exported:   61
Records restored:   61
ID matches:         61/61
Payload matches:    61/61
Vector matches:     61/61
Parity:             61/61 ✓

Zero-amnesia migration — every record survived the export → map → restore cycle.

Honest Limitations

• Float drift is real. Vector equality checks need an epsilon. I used np.allclose with atol=1e-6; if you need bit-exact vectors, store the raw bytes in the bundle instead.
• Payloads are only as portable as your mapper. If your source payload has nested objects with non-string keys (rare, but possible in JSON), the mapper needs explicit handling.
• This validates structure, not semantics. A round trip proves the bytes survived, not that a downstream agent will answer the same. That's a follow-up test I'd like to add (embed a probe question, compare answers before/after).
• The PR is awaiting maintainer review — the migration showcase repo pattern means multiple independent submissions are welcome, so the adapter stands on its own as a reference implementation either way.

What I'd Do Differently

1. Add semantic parity tests — same question, same answer, before and after migration
2. Support batch/streaming export for collections with millions of points (my exporter pages fine, but the mapper is single-threaded)
3. Auto-generate the field map from a sample of payloads instead of hand-configuring it

Why Bother With Open Memory Formats?

Lock-in is the quiet tax on agent development. Every memory system has its own collection schema, and the cost of switching grows with every record you store. An open format with working migration adapters means:

• You can switch vector DBs without rebuilding your agent's history
• You can audit what your agent actually remembers
• Your memory outlives any single vendor or tool

The adapter pattern (exporter → mapper → validator) is portable to any store. Same skeleton, different connector.

---

This was built as a contribution to the Memanto open memory format project. The full adapter implementation and test suite are in the migration PR.