Beyond Unit Tests: How to Test and Evaluate LLM Applications

Saurabh — Tue, 13 Oct 2026 15:12:21 +0000

Beyond Unit Tests: How to Test and Evaluate LLM Applications

Imagine deploying an AI-powered chatbot. Your API returns 200 OK, all your unit tests pass, and the application responds within an acceptable time.

But there is one problem: the answer is wrong.

The chatbot confidently provides an incorrect refund policy, invents a step that does not exist in the documentation, or answers a question that it does not have enough information to answer.

From a traditional software testing perspective, everything looks fine. From the user's perspective, the application has failed.

This is one of the biggest challenges when building applications powered by Large Language Models (LLMs).

Unlike traditional functions, LLMs can generate different responses to the same input. A response can be grammatically correct and well-structured while still being factually incorrect.

So, how do we test these applications effectively?

In this article, we'll explore practical approaches to LLM evaluation, hallucination testing, regression testing, and production monitoring, with examples relevant to JavaScript and backend engineers.

1. Why Traditional Unit Tests Are Not Enough

Consider a traditional function:

function calculateDiscount(price, discountPercentage) {
  return price - (price * discountPercentage) / 100;
}

console.log(calculateDiscount(1000, 10)); // 900

For a given input, we can define an expected output and assert that the result matches it.

import assert from "node:assert/strict";

assert.equal(calculateDiscount(1000, 10), 900);

This is a straightforward and deterministic test.

Now consider an AI assistant that answers questions using internal company documentation.

Question:

"What is the refund processing time?"

Expected answer:

"Refunds are processed within five business days."

The LLM might answer:

"Refunds are processed within five business days."
"You should receive your refund within five working days."
"The refund process typically takes about a week."

These answers may not be equivalent in every context. The third answer, for example, is less precise and might introduce ambiguity.

An exact string comparison cannot reliably tell us whether the response preserves the intended meaning.

This is why AI application testing needs more than traditional unit tests.

Traditional testing	AI evaluation
Does the function return the expected value?	Does the answer communicate the correct meaning?
Does the API return a successful status?	Does the response actually solve the user's problem?
Does the output match a fixed structure?	Is the response relevant, accurate, and supported by evidence?
Does the application handle errors?	Does it also behave safely when the answer is unknown?

The solution is not to replace unit tests. It's to combine them with additional evaluation techniques.

2. What Should We Measure in an AI Application?

Before evaluating an AI system, we need to define what a good response looks like.

There is no single metric that can measure every aspect of AI quality. Different applications require different evaluation criteria.

A. Correctness

Does the response provide the correct answer?

For a customer support assistant, correctness might mean that refund periods, eligibility conditions, and other policy details match the approved source.

Possible approaches include:

Comparing answers against reference answers.
Checking required facts or claims.
Having domain experts review a representative sample.
Using an automated evaluator with clearly defined criteria.

B. Relevance

Does the response answer the question that was asked?

For example, if someone asks how to reset a password, a detailed explanation of account registration is not a relevant answer, even if all the information is accurate.

Relevance matters when building documentation assistants, support chatbots, and search applications.

C. Faithfulness and hallucinations

For applications that retrieve documents before generating an answer, faithfulness means that the answer's claims are supported by the retrieved context.

Suppose the source document states:

"Refunds are processed within five business days."

The assistant answers:

"Refunds are processed within five business days, and a confirmation email is always sent within one hour."

If the source says nothing about a confirmation email, the additional claim is unsupported.

This is a useful way to test hallucinations in Retrieval-Augmented Generation (RAG) applications.

However, an answer that is faithful to a document is not automatically correct in the real world. The source itself could be outdated or incorrect, so source quality matters too.

D. Completeness

Does the response cover all the important parts of the question?

If a user asks for three steps and the assistant provides only two, the answer may be accurate but incomplete.

E. Safety

Does the application avoid unsafe or unauthorized behavior?

Examples include refusing to reveal another customer's private information, following access restrictions, and resisting attempts to bypass system instructions.

Safety tests should cover both ordinary usage and adversarial inputs.

The important lesson: Define your evaluation criteria before choosing the metrics. A high score on one metric does not guarantee that the application is good overall.

3. Building a Small Evaluation Dataset

One of the most practical ways to test an LLM application is to maintain a fixed dataset of representative questions and expected behavior.

This dataset can be treated as an evaluation suite and run whenever you change a prompt, model, retrieval configuration, or important application component.

For example:

Test scenario	Expected behavior
Ask for the refund processing time	Return the documented time
Ask about an unsupported policy	Acknowledge that the information is unavailable
Request a short summary	Preserve the key facts without adding claims
Ask a question unrelated to the documentation	Avoid inventing an answer
Request another customer's private data	Do not disclose unauthorized information

A basic dataset might look like this:

const evaluationCases = [
  {
    name: "Refund processing time",
    question: "How long does a refund take?",
    requiredPhrases: ["five business days"],
  },
  {
    name: "Unknown information",
    question: "What is the refund policy for a product not covered by the documentation?",
    uncertaintyPhrases: [
      "not available",
      "not specified",
      "cannot confirm",
      "don't have enough information",
    ],
  },
];

The required phrases and accepted uncertainty phrases are illustrative. They should be aligned with the actual policy and the response conventions used by your application.

As the application grows, include more examples covering different user intents, edge cases, and failure modes. Keep the dataset representative rather than filling it only with easy questions.

4. Writing AI Regression Tests in JavaScript

Let's turn the dataset into a simple automated test suite.

Assume that our application exposes an asynchronous answerQuestion() function that returns an object containing an answer property.

We can use Node.js's built-in test runner, avoiding the need for an additional test framework.

import test from "node:test";
import assert from "node:assert/strict";
import { answerQuestion } from "../src/assistant.js";

const evaluationCases = [
  {
    name: "Refund processing time",
    question: "How long does a refund take?",
    requiredPhrases: ["five business days"],
  },
  {
    name: "Unknown information",
    question: "What is the refund policy for an unsupported product?",
    uncertaintyPhrases: [
      "not available",
      "not specified",
      "cannot confirm",
      "don't have enough information",
    ],
  },
];

for (const scenario of evaluationCases) {
  test(scenario.name, async () => {
    const response = await answerQuestion(scenario.question);

    assert.equal(typeof response.answer, "string");
    assert.ok(response.answer.trim().length > 0);

    const answer = response.answer.toLowerCase();

    if (scenario.requiredPhrases) {
      for (const phrase of scenario.requiredPhrases) {
        assert.ok(
          answer.includes(phrase.toLowerCase()),
          `Expected answer to contain: ${phrase}`
        );
      }
    }

    if (scenario.uncertaintyPhrases) {
      const acknowledgesUncertainty =
        scenario.uncertaintyPhrases.some((phrase) =>
          answer.includes(phrase)
        );

      assert.ok(
        acknowledgesUncertainty,
        "Expected the assistant to acknowledge missing information"
      );
    }
  });
}

Run the tests with:

node --test

This is a simple baseline, not a complete semantic evaluator.

The code checks whether the response contains expected wording. That can catch obvious regressions, but a correct answer that uses different wording might fail. Conversely, a response could include the required phrase and still contain incorrect additional claims.

For production systems, combine these checks with claim-level verification, semantic evaluation, and human review where appropriate.

What happens when the prompt changes?

Imagine that the team modifies the system prompt to make answers shorter.

The change may improve response time and reduce token usage, but it could also remove important policy details.

Running the evaluation dataset again helps determine whether the change improved the system overall or introduced regressions.

This is the central benefit of an AI regression suite: changes are evaluated against a repeatable set of expectations instead of relying only on a few manual experiments.

5. Using an LLM as an Evaluator

Exact phrase matching is useful for specific requirements, but many answers need semantic evaluation.

For example, these two answers communicate approximately the same information:

"The request will be processed within five business days."
"Please allow five working days for processing."

A simple string comparison may mark them as different even when both are acceptable.

One approach is to use a separate LLM to evaluate the generated response. This technique is commonly known as LLM-as-a-judge.

The evaluator can be given:

The user's question.
The relevant source context.
The generated answer.
A rubric describing how the answer should be assessed.

For example, we might ask the evaluator to score each dimension from 1 to 5:

Dimension	Evaluation question
Relevance	Does the answer address the user's question?
Faithfulness	Are the answer's claims supported by the supplied context?
Completeness	Does it cover the required information?
Clarity	Is it understandable and unambiguous?

A simple evaluation instruction could be:

You are evaluating an AI-generated answer.

Question:
{question}

Source context:
{context}

Generated answer:
{answer}

Evaluate relevance, faithfulness, and completeness
on a scale from 1 to 5.

Use the supplied source context to assess faithfulness.
Do not reward an answer merely because it sounds confident.
Identify any unsupported claims and explain the reason
for each score.

Return structured JSON containing the scores,
unsupported claims, and a short explanation.

The evaluator should follow a consistent rubric and produce structured output that can be stored and compared.

However, an LLM evaluator is not a perfect source of truth. It can overlook errors, prefer fluent answers, or make its own incorrect judgments.

To improve reliability:

Validate the evaluator's results against human-reviewed examples.
Keep the evaluation instructions consistent across comparisons.
Use trusted source material where available.
Review borderline cases and disagreements.
Do not rely on one automated score for critical safety decisions.

LLM-as-a-judge is best treated as a scalable evaluation aid, not a replacement for ground truth or human judgment.

6. Measuring Latency, Token Usage, and Cost

Answer quality is only one part of production readiness.

A response can be accurate but take too long to generate. Another response might be acceptable but consume significantly more tokens than expected.

For this reason, quality evaluations should also track operational metrics.

Latency

Measure how long a request takes to complete.

Useful metrics include:

Median latency (p50).
p95 latency.
Timeout rate.
Error rate.

The p95 latency tells us the response time at or below which approximately 95% of measured requests complete. It helps identify slow experiences that an average can hide.

Token usage and cost

When supported by the model provider, collect input and output token counts for every request.

A simplified cost calculation is:

function estimateCost({
  inputTokens,
  outputTokens,
  inputPricePerMillion,
  outputPricePerMillion,
}) {
  return (
    (inputTokens / 1_000_000) * inputPricePerMillion +
    (outputTokens / 1_000_000) * outputPricePerMillion
  );
}

The price values should come from the applicable model's current pricing configuration. This calculation is an estimate and may not include other infrastructure or service costs.

Track cost per request, cost by feature, and cost per successfully completed user task.

The last measure is particularly useful because reducing the cost of each response does not necessarily improve the system if the answers become less useful.

Quality alongside performance

Consider a new model or prompt configuration that reduces average latency by 20% but also increases incorrect answers.

Looking only at latency would suggest an improvement. Evaluating quality and operational metrics together gives the team a more complete picture.

7. Monitoring an AI Application in Production

Offline evaluation helps us test changes before deployment. It cannot predict every real-world input or failure.

Production monitoring is therefore equally important.

A useful monitoring strategy includes:

Technical health

Request volume and error rate.
Model-provider failures and timeouts.
Median and p95 latency.

Cost and usage

Input and output tokens.
Estimated cost by model and feature.
Unexpected usage spikes.

AI quality

Periodic evaluation of sampled responses.
Unsupported claims and retrieval failures.
User feedback and task completion rates.
Changes in quality after a model or prompt update.

For a RAG system, it can also be valuable to monitor retrieval quality separately from answer generation. The retriever may return irrelevant documents even when the LLM is following those documents correctly.

When logging requests, take care to protect customer information. Redact sensitive data, apply appropriate access controls, and define retention rules rather than storing every raw prompt and answer indefinitely.

The goal is not to capture everything. It is to collect the right signals to detect failures and investigate them safely.

8. Putting It All Together: An AI Testing Workflow

A practical workflow for an engineering team could look like this:

Define quality criteria. Decide what a correct, relevant, safe response means for the feature.
Build an evaluation dataset. Include normal scenarios, edge cases, and known failure modes.
Run traditional tests. Verify API contracts, validation, authentication, error handling, and application logic.
Run AI evaluations. Check correctness, relevance, faithfulness, and the behaviors that matter to the feature.
Compare with the previous version. Investigate regressions after prompt, model, or retrieval changes.
Set deployment gates. Use thresholds for critical metrics that reflect the application's actual requirements.
Monitor production. Track latency, errors, cost, feedback, and sampled quality evaluations.
Feed real failures back into the dataset. Turn recurring production problems into regression tests.

This creates a continuous improvement loop: evaluation informs deployment, and production observations inform future evaluation.

Conclusion

Testing AI applications requires a broader approach than checking whether an API responds successfully.

Traditional unit and integration tests remain essential, but they do not establish that an LLM's answer is correct, relevant, complete, or safe.

A stronger strategy combines repeatable evaluation datasets, clearly defined quality criteria, semantic evaluation, regression testing, and production monitoring.

You do not need a complicated evaluation platform to get started. A small dataset, an automated test runner, a consistent evaluation rubric, and basic latency and cost tracking can already make AI development more disciplined.

The key takeaway is simple: don't just test whether your AI application works. Test whether it works well enough for the user and the problem it is designed to solve.

DEV Community: Saurabh

Beyond Unit Tests: How to Test and Evaluate LLM Applications

Beyond Unit Tests: How to Test and Evaluate LLM Applications

1. Why Traditional Unit Tests Are Not Enough

2. What Should We Measure in an AI Application?

A. Correctness

B. Relevance

C. Faithfulness and hallucinations

D. Completeness

E. Safety

3. Building a Small Evaluation Dataset

4. Writing AI Regression Tests in JavaScript

What happens when the prompt changes?

5. Using an LLM as an Evaluator

6. Measuring Latency, Token Usage, and Cost

Latency

Token usage and cost

Quality alongside performance

7. Monitoring an AI Application in Production

8. Putting It All Together: An AI Testing Workflow

Conclusion

Further Reading