Spring AI Evals: how I test agent behavior

Part 2: https://dev.to/lbobylev/baseline-for-spring-ai-evals-catching-silent-degradation-4ll6

When building an AI agent, there is usually a moment when the prompt seems to work. But as development continues, this can quickly get out of control. Today the agent can call the right tool. Tomorrow, after a small system prompt change, it can stop calling it. Later, it can start calling it when it should not. So I treat evals as regular regression tests for AI behavior.

Here is a small example: a Spring AI agent must create a Markdown note through the create_note tool. The task looks simple, but there are already several ways the agent can fail:

• not call the tool when needed, or call it when it is not needed;
• create an extra file;
• produce a note in the wrong format;
• add extra advice or any irrelevant content.

So I split the checks into a few layers.

Success Criteria

First I define what correct behavior means. The agent should:

• call create_note when the user asks to create a new note;
• not call create_note when the user asks to read, delete, rename, or explain;
• create only note.txt;
• put a heading on the first line;
• write short, relevant, and readable content.

This matters: without clear success criteria, an eval becomes a subjective check.

Tool Definition

The tool's scope is intentionally narrow: creating the note.txt file.

@Tool(
    name = "create_note",
    description = "Create a new Markdown note in note.txt. Use this only when the user asks to create a new note."
)
public String createNote(String title, String body) {
    var notePath = this.notesDirectory.resolve("note.txt");

    var content = "# " + normalize(title, "New Note")
            + System.lineSeparator()
            + System.lineSeparator()
            + normalize(body, "No content provided.")
            + System.lineSeparator();

    try {
        Files.createDirectories(this.notesDirectory);
        Files.writeString(notePath, content, CREATE, TRUNCATE_EXISTING);
        return "Created note.txt.";
    }
    catch (IOException ex) {
        return "Failed to create note.txt: " + ex.getMessage();
    }
}

The model uses the name and description input parameters to determine whether it needs to call the tool.

Agent Instructions

The process goals are formulated in the system prompt, which establishes the contract:

• when to use the tool;
• which file must appear;
• what must not appear in the result.

return this.chatClient.prompt()
        .system("""
            When the user asks to create a new note, use the create_note tool.
            The tool must create note.txt, put a Markdown heading on the first line,
            add the body text, and create no extra files.
            Keep the note concise and do not add unrelated suggestions.
            """)
        .user(message)
        .call()
        .content();

Deterministic Checks

The first layer checks tool invocation. This is no different from a normal unit test.

@ParameterizedTest
@CsvSource(value = {
    "note-create-01|true|Create a new note about my plans for the week",
    "note-create-02|true|Make a markdown note with the title Retro and the body: what to improve",
    "note-no-trigger-01|false|Show me the existing notes",
    "note-no-trigger-02|false|Read the note.txt file",
    "note-no-trigger-03|false|Delete the note.txt file",
    "note-no-trigger-04|false|Draft the text of a note, but do not create a file"
}, delimiter = '|')
void evaluatesWhetherAgentCallsNoteTool(String id, boolean shouldTrigger, String prompt) {
    doReturn("Created note.txt.").when(this.noteTool).createNote(anyString(), anyString());
    clearInvocations(this.noteTool);

    var response = this.chatController.chat(prompt);

    assertThat(response).as(id).isNotBlank();

    verify(this.noteTool, shouldTrigger ? atLeastOnce() : never()).createNote(anyString(), anyString());
}

Tool Unit Test

Next, I test the tool itself without the model. This is also a normal unit test.

@Test
void createNoteWritesOnlyNoteFileWithMarkdownHeading() throws IOException {
    var noteTool = new NoteTool(this.tempDir);

    var result = noteTool.createNote("Meeting Notes", "Discuss the evaluation setup.");

    var notePath = this.tempDir.resolve("note.txt");
    var content = Files.readString(notePath);
    var lines = Files.readAllLines(notePath);

    try (var fileStream = Files.list(this.tempDir)) {
        var files = fileStream.toList();

        assertThat(result).isEqualTo("Created note.txt.");
        assertThat(notePath).exists();
        assertThat(content).isNotBlank();
        assertThat(lines.getFirst()).startsWith("#");
        assertThat(files).containsExactly(notePath);
    }
}

This separates responsibilities: if the unit test fails, the problem is in the tool implementation rather than the agent's behavior.

Rubric-Based Assessment

Deterministic checks are good at catching facts, but not everything can be checked this way. For example, "the note is relevant," "there is no extra information," and "the text is easy to read" are qualitative requirements. Rubric-based assessment is used for these. In Spring AI, this is conveniently implemented with Evaluator: it receives an EvaluationRequest, runs a grader prompt inside, and returns an EvaluationResponse with pass, score, feedback, and metadata. The grader model is fixed, ideally using a versioned model name. For example, gpt-5.5-2026-04-23 instead of gpt-5.5. If the provider updates a model alias, the grader can become stricter or more lenient without any code change. In that case, a score change may be caused by grader drift rather than agent behavior. A low temperature is also preferable for grading. The grader should not be creative; it should apply the same rubric consistently. For rubric-based assessment, the grader runs multiple times. The pass rate better reflects its behavior.

class NoteStyleEvaluator implements Evaluator {

    private static final List<String> REQUIRED_CHECK_IDS =
            List.of("title", "relevance", "simplicity", "readability");

    private final ChatClient chatClient;

    private final int minimumScore;

    NoteStyleEvaluator(ChatClient.Builder chatClientBuilder, int minimumScore) {
        this.chatClient = chatClientBuilder.build();
        this.minimumScore = minimumScore;
    }

    @Override
    public EvaluationResponse evaluate(EvaluationRequest evaluationRequest) {
        var rubricResult = gradeNote(evaluationRequest);

        if (rubricResult == null) {
            return new EvaluationResponse(false, 0.0f, "No rubric result", Map.of());
        }

        var pass = rubricResult.overallPass()
                && rubricResult.score() >= this.minimumScore
                && hasRequiredPassingChecks(rubricResult.checks());

        var metadata = Map.<String, Object>of(
                "overallPass", rubricResult.overallPass(),
                "checks", rubricResult.checks());

        return new EvaluationResponse(pass, rubricResult.score(), rubricResult.toString(), metadata);
    }

    private StyleRubricResult gradeNote(EvaluationRequest evaluationRequest) {
        var graderPrompt = """
            Read the original user request and the note.txt content,
            then grade the note against these requirements:

            - it has a clear title;
            - the content is relevant to the original user request;
            - there is no clearly unnecessary information;
            - the text is easy to read.

            Return only a structured JSON object with these fields:
            - overall_pass: boolean
            - score: integer from 0 to 100
            - checks: exactly four items with ids title, relevance, simplicity, readability.

            Each check must include id, pass, and notes.

            original user request:
            %s

            note.txt content:
            %s
            """.formatted(
                evaluationRequest.getUserText(),
                evaluationRequest.getResponseContent()
            );

        return this.chatClient.prompt()
                .user(graderPrompt)
                .call()
                .entity(StyleRubricResult.class);
    }

    private boolean hasRequiredPassingChecks(List<StyleRubricCheck> checks) {
        if (checks == null || checks.size() != REQUIRED_CHECK_IDS.size()) {
            return false;
        }

        var checkIds = checks.stream()
                .map(StyleRubricCheck::id)
                .toList();

        return checkIds.containsAll(REQUIRED_CHECK_IDS)
                && checks.stream().allMatch(check ->
                        check.pass() && check.notes() != null && !check.notes().isBlank());
    }

    record StyleRubricResult(
            @JsonProperty("overall_pass") boolean overallPass,
            int score,
            List<StyleRubricCheck> checks) {
    }

    record StyleRubricCheck(
            String id,
            boolean pass,
            String notes) {
    }
}

Structured output enables automated verification. With Evaluation in Spring AI, grading becomes a normal Java contract:

• EvaluationRequest stores the user prompt and generated output;
• Evaluator contains the grading logic;
• EvaluationResponse returns a machine-checkable result;
• metadata stores details for each rubric check.

This makes grading convenient to integrate into CI: there is pass, score, feedback, and a list of checks.

@ParameterizedTest
@CsvSource(value = {
    "weekly-plan-01|Create a new note about my plans for the week. Include a clear title and only a concise list of concrete weekly goals.|80",
    "weekly-plan-02|Create a markdown note for my weekly planning. Mention only work, health, and learning goals. Do not add extra categories.|80"
}, delimiter = '|')
void gradesCreatedNoteWithStyleRubric(String id, String prompt, int minimumScore) throws IOException {
    var agentResponse = this.chatController.chat(prompt);

    assertThat(agentResponse).as(id).isNotBlank();
    assertThat(this.notePath).as(id).exists();

    var noteContent = Files.readString(this.notePath);

    var evaluator = new NoteStyleEvaluator(this.chatClientBuilder, minimumScore);

    int graderRuns = 3;
    int requiredPassingRuns = 2;

    var results = IntStream.range(0, graderRuns)
            .mapToObj(run -> evaluator.evaluate(new EvaluationRequest(prompt, noteContent)))
            .toList();

    long passingRuns = results.stream()
            .filter(EvaluationResponse::isPass)
            .count();

    assertThat(passingRuns)
            .as(id + ": " + results)
            .isGreaterThanOrEqualTo(requiredPassingRuns);
}

The agent runs once and creates one note. The grader then evaluates the same note.txt content three times; the test passes if at least two evaluations pass. This approach makes the grader result more robust against random model variation.

Saving Results Between Versions

It is also advisable to save eval results between runs. This helps compare behavior after changes to the prompt, model, or rubric. Pass/fail status is not always enough. A test can still pass while the score gradually gets worse. That is why I keep the score, feedback, and rubric metadata as artifacts.

var resultPath = Path.of("build", "eval-results", id + ".json");

var resultJson = objectMapper.writeValueAsString(Map.of(
        "id", id,
        "prompt", prompt,
        "minimumScore", minimumScore,
        "graderRuns", graderRuns,
        "requiredPassingRuns", requiredPassingRuns,
        "passingRuns", passingRuns,
        "results", results
));

Files.createDirectories(resultPath.getParent());
Files.writeString(resultPath, resultJson);

Separating Evals from Regular Tests

It is a good idea to keep AI evals separate from regular tests. They call the model API, so they are slower, more expensive, and less deterministic.

tasks.named('test') {
    useJUnitPlatform {
        excludeTags 'eval'
    }
}

tasks.register('evalTest', Test) {
    description = 'Runs AI agent evaluation tests.'
    group = 'verification'

    useJUnitPlatform {
        includeTags 'eval'
    }

    shouldRunAfter test
}

Run them separately:

./gradlew test
./gradlew evalTest

What This Gives

This makes development more predictable and manageable. When the system prompt, the tool description or implementation, or model parameters change, evaluating agent behavior remains much less subjective. CI tracks:

• whether the number of false positives increased;
• whether false negatives appeared;
• whether the expected artifact was created;
• whether the output passes the rubric assessment;
• whether quality dropped below the minimum score.

The most useful part is that the eval suite can grow gradually. Found a new miss? Add a new eval case. The agent failed on a real user prompt? That prompt becomes a regression test. This way, behavior drift becomes visible instead of staying hidden in manual testing.