Migrating a RAG Pipeline From One Vector Database Client to Another

#ai #machinelearning #llm #tutorial

The client swap itself is a morning’s work. What takes the rest of the week is the handful of places where the new client accepts your call, returns results, and means something different by them.

The five call sites that change

Grep your codebase for the vector store client and you will find that every use falls into one of five categories. Listing them first is worth doing, because it tells you how big the job is before you start and it stops you discovering a sixth in production.

Connection and index handle. Constructing a client, naming an index or collection. One or two lines, usually in a module that everything imports.
Index creation. Dimension, distance metric, index type and its build parameters. This is configuration, not code, but getting the metric wrong here is the most expensive mistake on the list because it invalidates everything written afterwards.
Writes. Upsert and delete. Batch size limits and the payload/metadata field name differ.
Reads. The similarity query, its top_k equivalent, whether vectors and metadata come back by default, and the filter.
Maintenance. Counting, listing identifiers, deleting by filter, and whatever your reindex job does. This is the surface people forget, and it is the one that breaks at 3am rather than in review.

The filter is large enough to be its own problem; why metadata filter syntax does not transfer covers the five syntaxes and the intents that have no translation. This page assumes you have solved that and concentrates on everything around it.

Identifiers and namespaces

Identifier types are the first thing that will refuse your data, and it is better to find out now than during the load. Pinecone record IDs are strings, and the pattern its own documentation uses for chunked documents is a compound string like doc1#chunk1. Qdrant point IDs are documented as an unsigned integer or a UUID — an arbitrary string is not a legal point ID there. If you are moving in that direction, every identifier in your corpus needs a mapping.

Do not allocate new random identifiers. Derive them deterministically, so that re-running the migration converges instead of duplicating, and so a bug report quoting an old ID is still traceable. A version-5 UUID over your existing string does both: same input, same UUID, forever. Keep the original string in the payload as well, because that is what your application logs, your citations and your delete-by-document logic all refer to.

import uuid

NS = uuid.UUID("6f8c4a1e-3d2b-4f9a-9c31-0b1d2e3f4a5b")  # fixed, checked in

def point_id(external_id: str) -> str:
    return str(uuid.uuid5(NS, external_id))

# "doc1#chunk1" -> stable UUID, every run, on every machine

The tenancy concept moves too. A Pinecone namespace is a partition inside one index and is passed per call; Qdrant has separate collections, and also supports many tenants in one collection separated by a payload field with an index on it. Those are not equivalent — the first is a hard partition you cannot query across, the second is a filter you can forget to apply. If your old code relied on the namespace argument being mandatory to keep tenants apart, the direct translation removes a safety property. Put the tenant filter somewhere it cannot be omitted: a wrapper that takes the tenant as a required argument and builds the filter itself, never a filter that callers assemble.

Rewriting the write path

The write is the most mechanical part. The shape is the same everywhere — an identifier, a vector, and a bag of metadata — and only the names change. Pinecone takes a list of records with id, values and metadata; the Qdrant Python client takes point structures with id, vector and payload.

# before
index.upsert(
    vectors=[
        {"id": "doc1#chunk1", "values": vec, "metadata": {"doc": "doc1", "year": 2024}},
    ],
    namespace="tenant-42",
)

# after
client.upsert(
    collection_name="chunks",
    points=[
        PointStruct(
            id=point_id("doc1#chunk1"),
            vector=vec,
            payload={"doc": "doc1", "year": 2024,
                     "tenant": "tenant-42", "external_id": "doc1#chunk1"},
        ),
    ],
)

Two things to carry over deliberately. First, batch sizes: request size limits differ and are expressed in bytes as often as in records, so a batch of 1,000 that was fine before may be rejected after. Size batches by serialised bytes rather than by count and you stop caring. Second, upsert semantics: check whether the destination replaces the whole record or merges the metadata. If it merges and you were relying on replacement, stale keys accumulate in the payload and your filters slowly start matching things they should not.

Rewriting the read path

The read has more implicit behaviour in it than the write, and the defaults are where the surprises live. Whether the stored vector comes back, whether metadata comes back, and how the result rows are shaped are all decisions the two clients make differently.

# before: metadata is opt-in, vectors are opt-in
res = index.query(
    vector=qvec, top_k=8, include_metadata=True,
    filter={"year": {"$gte": 2024}}, namespace="tenant-42",
)
hits = [(m["id"], m["score"], m["metadata"]) for m in res["matches"]]

# after: payload is on by default, vectors are not
res = client.query_points(
    collection_name="chunks", query=qvec, limit=8,
    query_filter=Filter(must=[
        FieldCondition(key="tenant", match=MatchValue(value="tenant-42")),
        FieldCondition(key="year", range=Range(gte=2024)),
    ]),
)
hits = [(p.payload["external_id"], p.score, p.payload) for p in res.points]

Note that the namespace argument became an ordinary filter condition. That is the tenancy change from earlier arriving in the code, and it is why the wrapper matters: there is now exactly one way to omit a tenant scope, and it compiles.

The other thing to check is what happens when the filter is very selective. A database that filters during the index traversal will return your full limit; one that filters after scanning an approximate index may return fewer rows than you asked for, or none, on a filter that matches only a small slice of the corpus. Test the read path with a filter that matches roughly one row in ten thousand and see what comes back. If the count collapses, that is not a bug in your translation — it is the destination’s filtering strategy, and it needs a configuration answer rather than a code one.

Scores do not mean the same thing

This is the one that ships. Every one of these systems returns a number called a score or a distance, and the direction of “better” is not consistent between them.

Cosine similarity runs from −1 to 1 and higher is better. Pinecone, Qdrant and Milvus all report cosine this way.
Cosine distance is one minus that, runs from 0 to 2, and lower is better. In Postgres, pgvector’s documented <=> operator is cosine distance, and <-> is L2 distance — both ascending.
Inner product is reported by pgvector as <#>, which the project’s README documents as the negative inner product, because Postgres only supports ascending index scans; you multiply by −1 to get the real value.
Euclidean distance is always ascending: lower is better, whichever system reports it.

Now consider the line of code almost every RAG pipeline contains: a relevance threshold, something like “drop anything below 0.75”. Move from a similarity to a distance and that comparison does not error — it silently inverts, keeping the worst results and discarding the best. The retrieved chunks are still ten chunks, still formatted correctly, still fed to the model. You will find out from the answers.

Fix it structurally rather than by flipping the sign. Convert whatever the client returns into one canonical quantity at the adapter boundary — similarity, ascending is worse — and let nothing downstream see the raw score. Then the threshold constant means one thing regardless of what is underneath it. And re-tune it anyway: a threshold calibrated on one embedding model and metric has no reason to hold on another. Vector similarity metrics covers what each metric is actually measuring.

Cutting over without a dark window

Do not swap the client and deploy. Run both, briefly, and compare.

Put both clients behind one interface with the five call sites above as its methods, and canonical scores at the boundary. This is a throwaway abstraction and that is fine; it exists for the length of the migration.
Load the destination from an export rather than by re-embedding. See exporting vectors and metadata — recomputing embeddings during a client migration means you are changing two things at once and will not be able to attribute a quality change to either.
Shadow-read: for some fraction of live queries, query both and log the two result ID lists. Do not use the new result. Compare overlap offline.
Investigate every query where overlap is below your bar. Expect the causes to be filter translation and approximate-index parameters, in that order.
Flip reads behind a flag, keep writes going to both, and leave the old store loaded until you have been through a full reindex cycle on the new one. The rollback you want is a flag, not a restore.
Stop dual writes only when you have a dated decision to. Dual-running forever is how you end up with two half-maintained stores.

Client method names, default result shapes and request size limits in this space change with major versions. Treat the calls above as the shape of the problem and check each against the current client reference before you copy them.

DEV Community