DEV Community

Lingesh B
Lingesh B

Posted on

Evaluating Amazon Bedrock Knowledge Base RAG using LLM-as-a-Judge

In this video, we perform a deep dive into evaluating Retrieval-Augmented Generation (RAG) performance using Amazon Bedrock Knowledge Bases and the LLM-as-a-Judge framework.

Top comments (1)

Collapse
 
alexshev profile image
Alex Shev

LLM-as-judge can be useful, but I would keep it as one layer, not the final truth. For RAG, the judge should inspect citation support, answer completeness, refusal behavior, and whether the retrieved context actually contains the claim.