Why do most RAG failures happen after retrieval? (Not where you'd expect)

I’ve been helping folks debug their RAG pipelines — some personal projects, some early-stage deployments.

At first, I thought the usual suspects were to blame: wrong embeddings, chunking too small, no overlap, etc.

But the more I look at it, the more I think many failures don’t happen at the retrieval step at all.

In fact, the chunk looks fine. Cosine similarity is high. The answer feels fluent. But it’s completely wrong — and not because the model is hallucinating randomly. It’s more like… the reasoning collapsed.

Here are some weird patterns I’ve started to see:

Retrieval hits the right doc, but misses the intended semantic boundary

Model grabs the right chunk, but interprets it in the wrong logical frame

Multiple chunks retrieved, but their context collides, leading to a wrong synthesis

Sometimes the first query fails silently if the vector DB isn't ready

Other times, the same input gives different results if called before/after warm-up

Have you run into this sort of thing? I’m trying to collect patterns and maybe map out the edge cases.

Would love to hear what others are seeing.
I’m not tied to any solution (yet 😅), just observing patterns and maybe overthinking it.