DEV Community

Ashwin Ugale profile picture

Ashwin Ugale

404 bio not found

Would your RAG eval suite notice if someone weakened the prompt?

Would your RAG eval suite notice if someone weakened the prompt?

6
Comments 6
3 min read

Want to connect with Ashwin Ugale?

Create an account to connect with Ashwin Ugale. You can also sign in below to proceed if you already have an account.

Already have an account? Sign in
I fault-injected two AI agent frameworks. One recovered — the other charged the card and said 'done'

I fault-injected two AI agent frameworks. One recovered — the other charged the card and said 'done'

8
Comments
6 min read
Your evals pass. That doesn't mean they work.

Your evals pass. That doesn't mean they work.

9
Comments 5
4 min read
Your agent ignored a failed tool call. Here's how to catch that in CI.

Determinism over LLM judges for CI gating

Your agent ignored a failed tool call. Here's how to catch that in CI.

17
Comments 41
5 min read
The reranker I added to improve RAG was causing most of my remaining misses

The reranker I added to improve RAG was causing most of my remaining misses

6
Comments 7
5 min read
Mutation testing, but for LLM evals — early experiment, would love feedback

Mutation testing, but for LLM evals — early experiment, would love feedback

6
Comments
2 min read
loading...