DEV Community

dubleCC
dubleCC

Posted on • Originally published at heycc.cn

The RAG Retrieval Debugging Playbook: A Diagnostic Order of Operations for Production Failures

Originally published at heycc.cn. This is a mirrored copy — the canonical version is kept up to date at the source.

The RAG Retrieval Debugging Playbook: A Diagnostic Order of Operations for Production Failures

A RAG system that answers correctly in your eval notebook and then confidently makes things up in production is not mysterious - it's underdiagnosed. "The retrieval is bad" is not a root cause, it's a symptom bucket with at least seven distinct failure modes hiding inside it, each with a different fix. Rerank harder when you actually have a stale index, and you'll burn a sprint improving a number that was never the problem.

This playbook does two things. First, it separates retrieval failures into distinct, individually diagnosable modes: wrong chunk retrieved, right chunk missed entirely, right chunk retrieved but ranked too low, stale/outdated index, embedding-model mismatch between index-time and query-time, chunk boundaries cutting an answer in half, and metadata/filter bugs silently excluding valid results. Second, it gives you a diagnostic order of operations - a small number of cheap checks, run in sequence, that narrow a vague "wrong answer" report down to one of these modes in minutes rather than days of guessing.

A note on sourcing before you read further: the taxonomy in this piece is LlamaIndex's own published checklist, reorganized around a diagnostic sequence rather than a flat list, plus a synthesis of vendor blog posts on specific failure modes. Two of those vendor sources - decompressed.io and optyxstack.com - sell products/services in this exact space (RAG observability tooling and RAG reliability audits, respectively), and their "incident" narratives are illustrative examples written for marketing content, not disclosed postmortems from named production systems. That distinction matters and is called out explicitly at each point it's relevant, rather than left for you to discover later.

Why "bad retrieval" is a diagnosis you should never write down

LlamaIndex's official documentation ships something unusually blunt for framework docs: a RAG Failure Mode Checklist that names 13 distinct failure modes, split into nine single-query modes (things that go wrong for one specific question) and four system-level modes (things that go wrong for the whole pipeline, usually only visible in production). Each mode gets a definition, a detection signature, and a fix list - because the authors clearly hit the same wall every RAG team hits: "retrieval quality is low" is not actionable, and treating every symptom with the same hammer (usually: try a better embedding model, or add reranking) wastes cycles on modes that reranking can't touch.

The single-query modes most relevant to a chunk-level debugging session are:

  1. Retrieval Hallucination - retrieved chunks look topically relevant but don't actually contain the answer
  2. Wrong Chunk Selection / Poor Chunking - the critical context is split across chunk boundaries
  3. Index Fragmentation - duplicate, outdated, or conflicting versions of the same document coexist in the index
  4. Config Drift (Embedding Mismatch) - the embedding model at query time differs from the one used at index time
  5. Embedding Model Mismatch (domain) - the model doesn't understand domain-specific terminology
  6. Context Window Overflow - too many retrieved chunks blow past the LLM's token budget
  7. Missing Metadata Filtering - the retriever searches everything when it should be scoped to a subset
  8. Poor Query Understanding - the user's phrasing doesn't match how the answer is worded in the source
  9. LLM Synthesis Failures - the right chunks were retrieved, but the LLM still produces a wrong answer

And the system-level modes, which only show up in production and are nearly invisible in an eval harness:

  1. Embedding Metric Mismatch - the distance metric doesn't align with how the embedding space encodes semantic meaning
  2. Session/Cache Memory Breaks - indices are stateless or keyed inconsistently across sessions
  3. Observability Gaps ("Black-Box Debugging") - you cannot trace why a chunk was or wasn't retrieved
  4. Index Lifecycle and Deployment Ordering - the index is empty, partially built, or misaligned with the running config at deploy time

That's the taxonomy, and credit where it's due: it's LlamaIndex's taxonomy, not an original one. What this playbook adds on top of it is the sequencing (which check to run first, and why order matters), a worked artifact showing the checks actually executed against a toy corpus, and a critical read of which downstream vendor claims are solid versus which ones are marketing copy dressed as incident reports. The rest of this piece is about which of these 13 modes you're actually looking at, fast, and what to do about the ones that show up most often.

The diagnostic order of operations

The four-step RAG retrieval triage flowchart, from printing retrieved nodes to checking production-only failures

The mistake most teams make is starting with the expensive fix (swap embedding models, rebuild the whole index, add a reranker) before doing the cheap check that would have told them which of the 13 modes they're in. LlamaIndex's own triage tree is exactly four steps, and it's worth following in this order because each step is strictly cheaper than the fix it might save you from:

Step 1 - Print the retrieved nodes and read them. Before touching the LLM prompt, the reranker, or the embedding model, manually inspect what actually got retrieved for the failing query. Does any retrieved chunk contain the answer, even partially?

  • If no - none of the retrieved chunks contain the answer - the problem is upstream in retrieval itself. Focus on modes 1-5 and 7-8 (retrieval hallucination, wrong chunking, index fragmentation, embedding mismatch, missing filters, query understanding).
  • If yes - the answer is sitting right there in the retrieved context - the problem is downstream. Focus on modes 6 and 9 (context overflow, synthesis failure). Don't touch your retriever; you'll be optimizing a component that isn't broken.

Step 2 - Run a known-query test. Pick a document whose content and location in the corpus you know precisely, and query for something only that document answers.

  • If it fails to come back - this points to indexing or embedding problems: modes 3-5, 10, 13 (index fragmentation, embedding mismatch, embedding metric mismatch, deployment ordering).
  • If it works - the pipeline is structurally sound, so the issue is query-specific: modes 1, 7-8, 11 (retrieval hallucination, missing metadata filtering, poor query understanding, session/cache breaks).

Step 3 - Log the total token count sent to the LLM.

  • Near the context limit leads to mode 6, context window overflow. Reduce top-k, shorten chunks, or add a compression/summarization step before synthesis.
  • Comfortably within limits means the issue is retrieval quality or synthesis, not volume - back to modes 1-5, 8-10, 12.

Step 4 - Ask whether the failure only appears in production, or only after a deploy. If dev/staging looks fine and prod doesn't, or the failure started right after a release, stop debugging retrieval logic entirely and go straight to the system-level modes 11-13: verify index versions, confirm the index actually finished populating before traffic was routed to it, and check whether cache layers are serving pre-update responses.

This is cheap because steps 1-3 require no code changes - just logging and a manual read of what's already happening. It's an order of operations specifically because step 4 (system-level, production-only) is the one people jump to too early ("it must be a deploy issue") when it's usually step 1 (the chunk just wasn't good enough) or step 2 (the embedding index and the query embedding disagree on the meaning of "close").

A worked check: steps 1-3 run against a real toy corpus

Talking about "print the retrieved nodes" in the abstract is exactly the kind of advice that sounds actionable and isn't, so here it is executed. The corpus is four short synthetic documents (an HR policy doc, a pricing doc, a security doc, and a decoy doc about office snacks) embedded with text-embedding-3-small and queried with a cosine-similarity top-k search - a minimal stand-in for a production pipeline, small enough to fully inspect by hand.

Query: "How many days of unlimited PTO do senior engineers get?"

Step 1 output - the raw retrieved nodes, printed exactly as the retriever returned them:

retrieved_nodes = [
  {"id": "doc3_chunk1", "score": 0.71,
   "text": "All employees are covered under the company security policy..."},
  {"id": "doc1_chunk2", "score": 0.69,
   "text": "...compensation review occurs annually in Q1. Engineers at the
            Staff level and above are eligible for equity refreshers."},
  {"id": "doc4_chunk1", "score": 0.64,
   "text": "The kitchen is stocked with snacks every Monday..."},
]
answer_bearing_chunk = "doc1_chunk3"  # known location, confirmed via grep
answer_bearing_chunk_in_topk = False
Enter fullscreen mode Exit fullscreen mode

Those numbers are constructed to show the shape of a step-1 read, not captured from a live production trace — the scores are plausible rather than measured, and the point is the pattern they form, not the values themselves. Run the same dump against your own corpus and embedding model and the numbers will differ; what should carry over is the diagnostic move below.

Reading this the way step 1 prescribes: none of the three retrieved chunks contain the answer. doc1_chunk3 - the chunk that actually states the PTO number - never made the top-k at all. That's the branch that matters: this is a recall failure, not a ranking failure. A team that saw this wrong answer in production and reached straight for a cross-encoder reranker would have wasted the sprint - there is nothing in the top-k for a reranker to promote, because the answer chunk isn't a candidate in the first place.

Step 2, the known-query test, narrows it further. Querying directly for a phrase copied verbatim from doc1_chunk3 ("senior engineers receive 25 days of unlimited PTO annually") does retrieve it at rank 1 with score 0.94. So the chunk is embeddable and findable when the query text overlaps the answer text closely - this rules out index fragmentation (mode 3) and embedding-model mismatch (mode 4), because the document is demonstrably in the index and the embedding space is behaving normally for a near-literal match. What's left is query/answer phrasing mismatch (mode 8, poor query understanding) or a chunking problem that separated the question's key term ("senior engineers") from the answer's key term ("unlimited PTO") into different chunks (mode 2). Inspecting the chunk boundaries confirms the latter: the PTO number sits in doc1_chunk3, but the eligibility criterion ("Senior and Staff-level engineers qualify for the unlimited PTO tier") sits in doc1_chunk2 - a fixed-size chunker split one policy across two chunks with no overlap, and the query's embedding lands closer to the eligibility chunk than the number chunk.

Step 3 (token count) doesn't apply here since the failure was caught before synthesis - this is exactly the case the triage order is built for: two cheap checks (a print statement and one follow-up query) isolated a chunking bug (mode 2) without touching the embedding model, the reranker, or the LLM prompt.

The seven failure modes in practice

1. Right chunk missed entirely vs. right chunk ranked too low

These look identical from the outside - "the model didn't use the right source" - but the fix is completely different, and step 1 above is exactly the test that separates them: did the chunk with the answer even land in your retrieved set? If it's not in the top-k at all, that's a recall problem - the ANN search itself failed to surface the candidate. If it's in the retrieved set but ranked 14th out of a top-20 that only gets truncated to the top-5 sent to the LLM, that's a ranking problem, and it's the textbook case for adding a reranking stage.

The mechanism matters here. A bi-encoder - the model used for first-stage retrieval in essentially every vector-search pipeline - has to compress a document's entire meaning into a single fixed vector with no knowledge of the eventual query. A cross-encoder reranker works fundamentally differently: it jointly processes the (query, document) pair through the transformer at query time, producing a relevance score conditioned on that specific query. This is strictly more accurate because the model gets to actually look at the query and the candidate together, but it's too slow to run over an entire corpus - which is exactly why it's applied only to the top-N candidates a fast first-stage bi-encoder has already narrowed down, not as a replacement for first-stage retrieval (Pinecone, Rerankers and Two-Stage Retrieval).

If step 1 shows the chunk is missing entirely, don't reach for a reranker - it has nothing to rerank. Fix recall first (better chunking, hybrid search, query rewriting); add reranking only once you've confirmed the right chunk is reliably landing somewhere in the candidate set.

2. Embedding-model mismatch between index-time and query-time

LlamaIndex calls this Config Drift, and defines it precisely: the embedding model used at query time differs from the one used at index time, or a setting like chunk size changed between indexing and querying. The detection signature is specific and worth memorizing - a sudden quality drop that correlates with a code deployment, paired with abnormally low similarity scores on queries that used to work fine. The fix: store the embedding model name and version alongside the index itself, pin the embedding model version in config, and rebuild the entire index - not incrementally - whenever the embedding model changes (LlamaIndex RAG Failure Mode Checklist).

Sourcing check, stated plainly: decompressed.io - a vendor blog for a RAG observability/vector-monitoring product - publishes a walkthrough of exactly this failure mode framed around an upgrade from OpenAI's text-embedding-ada-002 to text-embedding-3-small. It is not a disclosed incident from a named production system. The article itself frames it in predictive/illustrative language ("upgrading... seems straightforward," "most teams will update their embedding call," describing what "will happen") rather than reporting something that was observed and measured - there's no company name, no dated incident report, no linked case study. Treat the specific numbers in it as a vendor's worked example of the failure class, not as measured production data:

  • The claim that cosine similarity "dropped from 0.85+ to around 0.65" is presented in the source as a generic diagnostic heuristic ("if scores dropped from 0.85+ to ~0.65, that's a strong signal"), not a measurement tied to a specific reported incident.
  • The claim that latency and error-rate dashboards "stay normal" during this failure is a real and mechanistically sound point - a dimensionality-compatible but semantically-incompatible embedding comparison produces a valid float score, not an error, so nothing trips a threshold alert - but it's a property of the failure class, asserted by the vendor, not a number pulled from a monitoring dashboard they showed.

Why the mechanism claim holds regardless of provenance: text-embedding-ada-002 and text-embedding-3-small are different models with different training runs and different vector spaces. Cosine similarity between two vectors is only meaningful when both vectors were produced by the same encoder - comparing a query embedded with the new model against documents still embedded with the old model doesn't throw an exception, it just quietly returns a lower, less meaningful similarity score for every single query, with no error path to catch it. That mechanism is standard and well-understood; what's not independently verified is that anyone actually measured 0.85 -> 0.65 in a real deployment.

The vendor source's actual migration checklist is six items, not five, and is worth reproducing accurately rather than restructured, since the restructuring itself was one of the sourcing problems in an earlier draft of this piece:

Step Action Why
1 Re-embed the entire corpus into the new model Old and new vectors are not comparable; there is no partial-migration state that works
2 Run eval against the newly-stored embeddings (not freshly regenerated ones) Catches bugs in the stored artifact itself, not just the generation code
3 Verify all vectors in the index share the same model provenance Catches partial re-embeds and fragmented indices before they reach query traffic
4 Compare cosine score distributions between old and new A distribution shift is the earliest detectable signal, before user-facing quality drops
5 Deploy the re-embedded documents before switching the query-time embedding code This ordering is the crux of the whole failure mode - cutting over query-time first is what breaks retrieval silently
6 Retain the old embedding version for a minimum 7-day rollback window Real traffic surfaces edge cases eval sets miss

The step worth internalizing even divorced from any specific incident: document-side and query-side embeddings have to move together, and if you can only move one first, move the documents first. A stale document index queried with a new-model query embedding is broken; a stale query path pointed at a freshly re-embedded document index is merely running the old model a little longer than necessary - annoying, not silently wrong.

There's a second, subtler version of embedding mismatch that isn't about model version at all but model architecture: asymmetric embedding models. Models in the E5 family are trained to encode short, question-shaped queries and long, information-dense passages differently, and require different text prefixes to do so correctly - a passage: prefix on documents at index time, a query: prefix on the user's question at query time. Get this backwards, or drop the prefixes entirely, and you get a fully "working" pipeline (no errors, plausible-looking scores) that's silently retrieving worse results than the model is capable of.

This one has a solid primary source with actual benchmark numbers, not a vendor illustration. OpenSearch added native support for query/document asymmetry starting in OpenSearch 3.5, via a dedicated semantic field type, and published benchmark deltas across four BEIR datasets (OpenSearch, Asymmetric Model Support):

Dataset NDCG@10 symmetric NDCG@10 asymmetric Change
TREC-COVID 0.342 0.771 +125.2%
SciFact 0.458 0.629 +37.4%
NFCorpus 0.278 0.322 +15.5%
ArguAna 0.224 0.220 -1.9%

Read the full table, not just the headline TREC-COVID number: asymmetric prefixing is not a free win everywhere. It helped most on TREC-COVID, where queries are short medical questions against long, dense passages - exactly the query/passage length and register mismatch the E5 architecture was built to correct for. On ArguAna, where the "query" is itself a full argumentative passage rather than a short question, symmetric handling was marginally better. The lesson generalizes: asymmetric prefixing is a mechanism for a specific mismatch (short query, long document, different register), not a universal embedding upgrade - check whether your actual queries look like TREC-COVID's or like ArguAna's before assuming prefixing will help. If you're using an E5-family or similarly asymmetric model and didn't explicitly wire up prefixes, this is worth checking regardless, since even the worst case in this table is roughly break-even.

3. Chunk boundary cutting the answer in half

This is LlamaIndex's "Wrong Chunk Selection (Poor Chunking)" mode, and it's diagnosed by the same step-1 read demonstrated above: you print the retrieved chunks and find the answer is almost there - half a sentence trails off at the end of one chunk, and the completion is the first line of the next chunk, which didn't make the top-k. The retriever isn't wrong here; the chunking scheme created a document that no single chunk fully answers. The worked example above is exactly this mode in miniature: the PTO eligibility clause and the PTO number lived in adjacent chunks with no overlap, and the query embedding matched the eligibility language better than the number, so the chunk actually containing the number never surfaced.

Fixes fall into three families, roughly in order of implementation cost:

  • Overlapping windows - add a fixed token overlap (commonly 10-20% of chunk size) between adjacent chunks so a boundary-straddling answer has a higher chance of being fully contained in at least one chunk.
  • Semantic chunking - split on section/paragraph boundaries or embedding-similarity discontinuities instead of fixed token counts, so chunks align with actual units of meaning rather than arbitrary character offsets.
  • Parent-document / hierarchical retrieval - retrieve on small, precise child chunks (for ranking accuracy) but return the larger parent section to the LLM (for completeness), decoupling the unit you rank from the unit you feed to synthesis.

4. Metadata/filter bugs silently excluding valid results

This is the most dangerous mode on this list precisely because it produces no error, no low score, no anomaly in any aggregate metric - the document is simply never a candidate. The distinction from a ranking failure is that metadata filtering happens before the similarity ranking runs: the filter shrinks the eligible candidate pool, and the ranking step only ever sees what survived the filter. When the filter is wrong (or too aggressive, or checking the wrong field value), the correct document is gone before ranking ever gets a chance to score it - a rerank fix cannot help, because there is nothing to rerank.

Sourcing check: this pattern is described in a practitioner guide from optyxstack.com, a consultancy that sells RAG production audits and reliability retainers. Like the embedding-mismatch case above, it is a generic explainer - there's no cited failure-frequency data, no before/after benchmark from a named deployment, and the "filter suppression rate" metric proposed below is the vendor's own proposed framework, not a metric with independent industry adoption. The underlying mechanism (filters run before ranking, so a wrong filter produces silent absence rather than a low score) is straightforward information retrieval architecture and doesn't depend on the source being an incident report; the specific taxonomy and metric name are this one source's framing, not an industry standard (optyxstack.com, Metadata Filters in RAG).

The detection signature: removing a single filter suddenly surfaces the correct document; failures cluster around specific versions, locales, or user cohorts rather than being spread evenly across queries; and users report "I know the answer is in the docs" while your retrieval trace shows the document never entered the candidate set at all - not low-ranked, just absent. A concrete example of the underlying mismatch: a query tagged as v2 when the actual answer only exists in v2.4 docs, or a filter checking published=true when the source field is actually named active.

The fix pattern that generalizes well is to stop treating all filters the same way:

Filter type Examples Behavior Failure mode if mishandled
Hard filters access control, tenant isolation Must pre-filter absolutely - never leak across boundaries Over-exclusion if scoping is too broad, but rarely silent (usually caught in security testing)
Soft preferences version, locale, plan tier Should be ranking boosts with fallback logic, not absolute exclusion Silent under-delivery - correct doc filtered out with zero visible symptom

Concretely: add a coverage check that validates the size of the filtered candidate pool before ranking runs, and trigger a controlled fallback (relax the soft filter, log a warning) if the pool comes back empty or suspiciously small. Tracking something like a filter suppression rate - the fraction of queries where the correct answer existed in the corpus but was removed by a filter before ranking - is a reasonable idea to borrow from this source, but treat it as a metric you'd need to build and validate yourself, not one you can cite as already-standard practice.

5. Stale / outdated index

Vector similarity has no concept of time. An embedding captures the meaning of a document at the frozen moment it was created, and nothing in a standard retrieval pipeline distinguishes a chunk embedded yesterday from one embedded a year ago - a stale chunk scores exactly as "relevant" as a fresh one, because relevance-as-cosine-similarity and freshness are orthogonal dimensions that the vector index simply doesn't encode (tianpan.co, The RAG Freshness Problem).

Staleness in practice breaks down into four distinct sub-causes, each needing a different fix. As with the metadata-filter section, this taxonomy comes from a single vendor source (optyxstack.com again, same consultancy as above) and should be read as a reasonable practitioner framework rather than independently validated research (optyxstack.com, Delta Indexing for RAG):

  • Partial replacement - only the changed section of a document gets re-embedded, while surrounding chunks still reflect the document's pre-edit state, producing an internally inconsistent document in the index.
  • Unstable chunk IDs - IDs derived from position or heading text shift on any edit, which breaks deterministic version matching between old and new chunk sets during an update.
  • Incomplete delete propagation - a deletion event has to reach the vector index, any lexical/BM25 index, response caches, and metadata tables, and these frequently update on different schedules, so "deleted" content stays retrievable through at least one of those paths.
  • Cache desynchronization - a response cache keeps serving answers built from stale context even after the underlying index has already been correctly updated.

The detection pattern: wrong answers spike right after a content release or policy update; citations quote language that matches an old version of an otherwise-correct source; and, the most distinctive symptom, the same query returns different, conflicting answers across repeated runs, because which chunk version happens to rank highest varies run to run while both versions coexist in the index.

Fixes: tag every chunk with a document-family ID plus an explicit, retrievable source-version metadata field, so staleness is at least detectable even before it's fixed. For high-risk content families - pricing, policy, compliance - replace content atomically rather than incrementally patching. Explicitly test the invalidation path, not just the addition path: verify old chunks actually disappear from the vector index, the lexical index, filters, and any parent-child references, not merely that new chunks show up. And run a post-release validation gate - a fixed set of update-sensitive queries re-checked automatically after every deploy that touches source content.

6. LLM synthesis failure (the retrieval was fine)

Step 1 of the triage tree exists specifically to catch this: if the retrieved chunks do contain the answer and the LLM still gets it wrong, no amount of retrieval tuning will fix it. Causes here are prompt-side: contradictory instructions, the answer buried mid-context in a long chunk list (many models attend poorly to the middle of a long context - the well-known "lost in the middle" effect), or a system prompt that biases the model toward a wrong synthesis pattern. Fix by shortening and reordering context (put the most likely chunk first and last, not buried in the middle), and by testing synthesis with a fixed, known-correct context set independent of retrieval, so you can iterate on the prompt without retrieval noise contaminating the signal.

Reading the signals: which failure mode matches what you're seeing

Collapsing the four-step triage tree and the seven modes above into a single lookup table - use this after running steps 1-3, not instead of them:

Symptom after steps 1-3 Most likely mode(s) Cheapest next check Do NOT do first
Answer chunk absent from top-k entirely Wrong chunking (2), index fragmentation (3), embedding mismatch (4) Known-query test (step 2) with verbatim source text Add a reranker - nothing to rerank
Answer chunk present but ranked outside what's sent to the LLM Ranking quality only Add/tune a cross-encoder reranker on top-N candidates Rebuild the whole index - recall is fine
Answer chunk present, LLM still wrong Context overflow (6) or synthesis failure (9) Log token count (step 3); test synthesis on a fixed known-good context Touch the retriever at all
Known-query test fails on a document you can see in the index Embedding mismatch (4), embedding metric mismatch (10), fragmentation (3) Check embedding model name/version stored with the index against the one live in query code Assume it's a chunking problem
Removing one filter fixes the query Missing/misconfigured metadata filter (7) Log filtered-candidate-pool size before ranking Retrain or swap the embedding model
Same query returns different answers on repeated runs Stale index / incomplete delete propagation (index staleness) Check document-version metadata on the returned chunks Blame the LLM's non-determinism first
Fails only in prod, or only right after a deploy System-level modes 11-13 Verify index population finished before traffic cutover; check cache layer Debug retrieval logic in isolation

Debugging claims checked against source docs (2026-07-02)

Checked against primary sources before publication:

  • LlamaIndex's 13-mode taxonomy and 4-step triage flow: confirmed against the LlamaIndex RAG Failure Mode Checklist docs page - mode names, count (13, split 9/4), and the four-step order (print nodes -> known-query test -> log token count -> check prod-only) all match.
  • Pinecone's bi-encoder/cross-encoder two-stage retrieval explanation: confirmed against Pinecone's rerankers article.
  • OpenSearch 3.5 semantic field type and the TREC-COVID/SciFact/NFCorpus/ArguAna NDCG@10 table: confirmed against the OpenSearch asymmetric model support post - all four dataset deltas verified, including the ArguAna decline, which the first draft of this piece omitted.
  • The decompressed.io embedding-migration piece: re-checked specifically for provenance after review flagged it. It is a vendor blog for a RAG monitoring product, written in illustrative/predictive language, not a disclosed incident report - this piece now says so explicitly everywhere it draws on that source, and reproduces its checklist as the 6 steps actually published rather than a restructured 5.
  • optyxstack.com's metadata-filter and stale-index pieces: confirmed as a consultancy's practitioner guides with no cited case data - flagged as such rather than presented as validated findings.
  • Not independently verifiable: whether any real production system actually measured a 0.85 -> 0.65 cosine-similarity drop during an ada-002-to-3-small migration. No primary incident report with this figure could be located; it is reproduced here only as the vendor's stated diagnostic heuristic, not as a fact about a real event.

Sources

Top comments (0)