DEV Community

Cover image for How to Test AI Responses Against Changing Customer Context
Virginia Nyambura  Mwega
Virginia Nyambura Mwega

Posted on Originally published at virginiamwegahashnodedev.hashnode.dev

How to Test AI Responses Against Changing Customer Context

Imagine your AI customer-support application generates this response:

Your refund has been completed.

Your evaluator checks the response and returns pass.

That sounds good. But there is one important question:

What was the verified refund status when the response was evaluated?

If the status was completed, the response is supported.

If the status was processing, the response is incorrect.

The generated sentence is identical. The evidence is different.

This is a testing problem that becomes important when AI applications generate responses using customer records, transaction data, order histories, or other changing application state.

An evaluator should not approve a response simply because it sounds reasonable. It needs to check whether the claims in that response agree with the relevant verified information.

In this article, I'll walk through a practical regression-testing approach I use in my AI Customer Support Response Evaluator.

1. Start with a realistic failure scenario

Suppose an application provides an AI assistant with the following verified customer information:

const customerContext = {
  refund_status: "processing"
};
Enter fullscreen mode Exit fullscreen mode

The assistant generates:

Your refund has been completed.
Enter fullscreen mode Exit fullscreen mode

A response-quality evaluator might recognize this as a clear and professional answer.

But a reliable factual evaluator needs to check the actual claim.

The response says the refund is completed. The verified context says it is still processing.

Those two pieces of information contradict each other.

The expected result should therefore be:

fail
Enter fullscreen mode Exit fullscreen mode

Now change only the verified status:

const customerContext = {
  refund_status: "completed"
};
Enter fullscreen mode Exit fullscreen mode

Keep the generated response exactly the same.

The expected result becomes:

pass
Enter fullscreen mode Exit fullscreen mode

This gives us a useful testing principle:

Hold the response constant and change the verified context. The evaluator's verdict should change when the evidence changes.

I use the term context-contrastive testing to describe this practical test-design approach.

2. Build a contrastive regression test matrix

Before writing tests, define the expected behavior.

For the refund example, the test matrix looks like this:

Response claim Verified status Expected result
Refund completed processing Fail
Refund completed completed Pass
Refund failed processing Fail
Refund failed failed Pass

This matrix covers four relationships:

  • A claim contradicted by the verified status.
  • A claim supported by the verified status.
  • Another contradicted claim.
  • Another supported claim.

Why test both directions?

Because an evaluator that rejects every response is not reliable. Neither is one that approves every response.

We need to establish that its decisions follow the evidence.

3. Turn the scenarios into regression tests

The following is an illustrative JavaScript example of how a test matrix could be structured. It demonstrates the test-design pattern, not the exact implementation or API of my evaluator.

const cases = [
  {
    response: "Your refund has been completed.",
    refundStatus: "processing",
    expected: "fail"
  },
  {
    response: "Your refund has been completed.",
    refundStatus: "completed",
    expected: "pass"
  },
  {
    response: "Your refund has failed.",
    refundStatus: "processing",
    expected: "fail"
  },
  {
    response: "Your refund has failed.",
    refundStatus: "failed",
    expected: "pass"
  }
];

describe("refund claim evaluation", () => {
  test.each(cases)(
    "returns $expected for refund status $refundStatus",
    async ({ response, refundStatus, expected }) => {
      const result = await evaluateSupportResponse({
        response,
        context: {
          refund_status: refundStatus
        }
      });

      expect(result.verdict).toBe(expected);
    }
  );
});
Enter fullscreen mode Exit fullscreen mode

The important part is the structure:

  1. Define the response.
  2. Supply the verified context.
  3. Define the expected verdict.
  4. Run the evaluator.
  5. Assert that the actual verdict matches the expected one.

The function name and result shape above are illustrative. In a real project, use the evaluator's actual interface and verdict schema.

4. What I implemented in my evaluator

This testing approach is already part of my AI Customer Support Response Evaluator.

I implemented four separate regression scenarios covering the refund-state relationships in the test matrix.

The tests check that:

  • A completion claim fails when the verified status is processing.
  • The same completion claim passes when the status is completed.
  • A failure claim fails when the status is processing.
  • The failure claim passes when the status is failed.

These tests are useful because they check more than whether the evaluator recognizes particular words.

They check whether the evaluator distinguishes between a claim and the verified state that supports or contradicts it.

They also give the project a concrete regression boundary: changes to evaluation logic should not silently break these expected decisions.

5. Expand beyond supported and contradicted claims

The four scenarios provide a foundation, but real applications have more complicated contexts.

I would extend this test strategy in three directions.

Test irrelevant context changes

Suppose the customer's preferred communication channel changes, but the refund status remains completed.

That unrelated change should not alter whether a refund-completion claim is supported.

This checks whether the evaluator is relying on relevant evidence rather than arbitrary context.

Test missing evidence

What happens when the refund status is unavailable?

The evaluator should not confidently approve the completion claim without evidence.

An unknown or equivalent outcome can be more appropriate than forcing an unsupported decision into pass or fail.

Missing evidence does not automatically prove that a claim is false.

Test conflicting evidence

Imagine a customer record contains multiple refund events or conflicting transaction information.

Which record should the evaluator use?

A reliable system needs a defensible way to identify the relevant transaction and handle ambiguity. It should not silently choose whichever record makes the generated response appear correct.

These are additional test dimensions to develop and verify. They are not claims about what the four existing refund regression tests already cover.

6. Apply the same method to other AI workflows

The pattern is not limited to refund support.

Application AI-generated claim Context to verify
Delivery support "Your order has been delivered." Verified delivery status
Payments "Your payment was successful." Transaction status
Subscriptions "Your subscription has been cancelled." Subscription state
Account support "Your account is verified." Verification record
Booking systems "Your appointment is confirmed." Booking status

For each workflow, identify the claims an AI assistant can make, the authoritative source of truth, and the context changes that should alter the expected verdict.

Then design tests around those relationships.

7. A checklist for context-aware AI evaluation

Before trusting an evaluator in a real application, ask:

  • Does it check claims against verified application data?
  • Can the same response pass in one context and fail in another?
  • Does irrelevant context leave the verdict unchanged?
  • Does it handle missing evidence without inventing certainty?
  • Can it distinguish conflicting records?
  • Are these behaviors protected by regression tests?

These questions help make evaluation more concrete and testable.

They also encourage a shift away from measuring only how good an AI response sounds.

Final takeaway

A successful AI response is not necessarily a correct AI response.

For context-aware applications, the evaluator must establish whether the generated claims are supported by the information the application has actually verified.

In my AI Customer Support Response Evaluator, testing the same refund claim against different verified refund states gave me a practical way to check this behavior.

The engineering principle is worth carrying into other AI applications:

Don't just test whether an AI response can pass. Test whether it knows when that same response should fail.

Further reading

Context-contrastive testing is the practical term used here for holding a generated response constant while varying the verified application context. The four refund-state scenarios described are implemented regression tests in my AI Customer Support Response Evaluator.

Top comments (0)