DEV Community

lbobylev
lbobylev

Posted on

Spring Boot 4 + Spring AI + Langfuse: Tracing LLM Calls and Tool Calling

#ai #springboot #java #langfuse

In this tutorial, we will build a small Java project using Spring Boot 4, Spring AI 2, Gradle, and OpenAI.

The application will:

  • respond to a normal HTTP request;
  • call an LLM through Spring AI;
  • use a WeatherTools tool that gets weather data from the internet;
  • send traces to Langfuse through OpenTelemetry;
  • add prompts and responses to Langfuse through a CallAdvisor.

Langfuse is a platform for monitoring and evaluating AI applications. The experience is very similar to LangSmith: you can trace LLM calls, inspect inputs and outputs, track latency and costs, and compare prompts and models. Langfuse is also open source and can be self-hosted, which makes it a good choice for teams that want more control over their data and infrastructure.

1. Project Stack

We use:

  • Java 21
  • Spring Boot 4.1.0
  • Spring AI 2.0.0
  • Gradle
  • OpenAI
  • Langfuse
  • OpenTelemetry / OTLP

The main dependencies in build.gradle:

dependencyManagement {
    imports {
        mavenBom 'org.springframework.ai:spring-ai-bom:2.0.0'
    }
}

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.boot:spring-boot-starter-opentelemetry'
    implementation 'org.springframework.ai:spring-ai-starter-model-openai'

    compileOnly 'io.opentelemetry:opentelemetry-exporter-otlp'

    testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
Enter fullscreen mode Exit fullscreen mode

2. Application Configuration

In application.properties, configure OpenAI, Spring AI observability, and the Langfuse OTLP endpoint:


spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-5
spring.ai.tools.observations.include-content=true
spring.ai.chat.client.observations.log-prompt=true
spring.ai.chat.client.observations.log-completion=true
spring.ai.chat.observations.log-prompt=true
spring.ai.chat.observations.log-completion=true

management.otlp.metrics.export.enabled=false
management.tracing.sampling.probability=1.0
management.tracing.export.otlp.enabled=${LANGFUSE_TRACING_ENABLED:false}
management.opentelemetry.tracing.export.otlp.endpoint=${LANGFUSE_OTLP_ENDPOINT:http://localhost:3000/api/public/otel}
management.opentelemetry.tracing.export.otlp.transport=http

langfuse.public-key=${LANGFUSE_PUBLIC_KEY:}
langfuse.secret-key=${LANGFUSE_SECRET_KEY:}
Enter fullscreen mode Exit fullscreen mode

The following setting is important:

management.otlp.metrics.export.enabled=false
Enter fullscreen mode Exit fullscreen mode

We use Langfuse for traces, not for Micrometer metrics.

If OTLP metrics export is not disabled, the application may try to send metrics to the same endpoint and receive a ConnectException.

3. Controller

The main controller is simple:

@RestController
class HelloAiController {

    private final ChatClient chatClient;
    private final WeatherTools weatherTools;

    HelloAiController(
            ChatClient.Builder chatClientBuilder,
            WeatherTools weatherTools,
            LangfusePromptTracingAdvisor langfusePromptTracingAdvisor
    ) {
        this.chatClient = chatClientBuilder
                .defaultAdvisors(langfusePromptTracingAdvisor)
                .build();

        this.weatherTools = weatherTools;
    }

    @GetMapping("/ai")
    String ai(
            @RequestParam(defaultValue = "Say hello in one short sentence")
            String message
    ) {
        return this.chatClient.prompt()
                .user(message)
                .tools(this.weatherTools)
                .call()
                .content();
    }
}
Enter fullscreen mode Exit fullscreen mode

What happens here:

  • GET /ai?message=... sends a prompt to the LLM.
  • .tools(this.weatherTools) connects the Java weather tool.
  • LangfusePromptTracingAdvisor adds the prompt and response to the Langfuse trace.

4. Weather Tool

The tool uses Open-Meteo:

  1. It finds the coordinates of a city.
  2. It gets the current weather using those coordinates.
  3. It returns the result as a normal string.
  4. The model uses the result to answer the user.

Spring AI allows Java methods to be exposed as model tools using @Tool.

@Component
class WeatherTools {

    private final RestClient restClient;

    WeatherTools(RestClient.Builder restClientBuilder) {
        this.restClient = restClientBuilder.build();
    }

    @Tool(description = "Get current weather for a city using live internet weather data")
    String getCurrentWeather(
            @ToolParam(description = "City name, for example Berlin or New York")
            String city
    ) {
        GeocodingResponse geocoding = this.restClient.get()
                .uri(
                        "https://geocoding-api.open-meteo.com/v1/search"
                                + "?name={city}&count=1&language=en&format=json",
                        city
                )
                .retrieve()
                .body(GeocodingResponse.class);

        if (geocoding == null
                || geocoding.results() == null
                || geocoding.results().isEmpty()) {
            return "Weather is unavailable: city not found: " + city;
        }

        Location location = geocoding.results().getFirst();

        ForecastResponse forecast = this.restClient.get()
                .uri(
                        "https://api.open-meteo.com/v1/forecast"
                                + "?latitude={latitude}"
                                + "&longitude={longitude}"
                                + "&current=temperature_2m,relative_humidity_2m,"
                                + "wind_speed_10m,weather_code",
                        location.latitude(),
                        location.longitude()
                )
                .retrieve()
                .body(ForecastResponse.class);

        if (forecast == null || forecast.current() == null) {
            return "Weather is unavailable for " + location.name();
        }

        CurrentWeather current = forecast.current();

        return "Current weather in %s, %s: %.1f C, humidity %d%%, "
                + "wind %.1f km/h, weather code %d."
                .formatted(
                        location.name(),
                        location.country(),
                        current.temperature2m(),
                        current.relativeHumidity2m(),
                        current.windSpeed10m(),
                        current.weatherCode()
                );
    }

    private record GeocodingResponse(List<Location> results) {
    }

    private record Location(
            String name,
            String country,
            double latitude,
            double longitude
    ) {
    }

    private record ForecastResponse(CurrentWeather current) {
    }

    private record CurrentWeather(
            @tools.jackson.annotation.JsonProperty("temperature_2m")
            double temperature2m,

            @tools.jackson.annotation.JsonProperty("relative_humidity_2m")
            int relativeHumidity2m,

            @tools.jackson.annotation.JsonProperty("wind_speed_10m")
            double windSpeed10m,

            @tools.jackson.annotation.JsonProperty("weather_code")
            int weatherCode
    ) {
    }
}
Enter fullscreen mode Exit fullscreen mode

5. Sending Traces to Langfuse Through OpenTelemetry

Langfuse accepts OpenTelemetry traces through an OTLP endpoint:

http://localhost:3000/api/public/otel
Enter fullscreen mode Exit fullscreen mode

Spring Boot sends traces through the spring-boot-starter-opentelemetry dependency.

Langfuse requires Basic Authentication using the public and secret keys. A separate configuration adds the required HTTP headers:

@Configuration
class LangfuseTracingConfig {

    @Bean
    OtlpHttpSpanExporterBuilderCustomizer langfuseOtlpHeaders(
            @Value("${langfuse.public-key:}") String publicKey,
            @Value("${langfuse.secret-key:}") String secretKey
    ) {
        return builder -> {
            if (!StringUtils.hasText(publicKey)
                    || !StringUtils.hasText(secretKey)) {
                return;
            }

            String token = Base64.getEncoder()
                    .encodeToString(
                            (publicKey + ":" + secretKey)
                                    .getBytes(StandardCharsets.UTF_8)
                    );

            builder.addHeader("Authorization", "Basic " + token);
            builder.addHeader("x-langfuse-ingestion-version", "4");
        };
    }
}
Enter fullscreen mode Exit fullscreen mode

Set the environment variables:

export LANGFUSE_TRACING_ENABLED=true
export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
Enter fullscreen mode Exit fullscreen mode

6. Adding Prompts to Langfuse Through CallAdvisor

Spring AI can log prompts and completions, but it does not always put them into Langfuse input and output fields in the format we need.

For this, we use the standard Spring AI CallAdvisor mechanism.

@Component
class LangfusePromptTracingAdvisor implements CallAdvisor {

    private final ObjectMapper objectMapper = new ObjectMapper();

    @Override
    public ChatClientResponse adviseCall(
            ChatClientRequest chatClientRequest,
            CallAdvisorChain callAdvisorChain
    ) {
        var span = Span.current();

        String input = serializePrompt(chatClientRequest);

        span.setAttribute("langfuse.observation.type", "generation");
        span.setAttribute("input.value", input);
        span.setAttribute("langfuse.observation.input", input);

        ChatClientResponse chatClientResponse =
                callAdvisorChain.nextCall(chatClientRequest);

        String output =
                serializeResponse(chatClientResponse.chatResponse());

        span.setAttribute("output.value", output);
        span.setAttribute("langfuse.observation.output", output);

        return chatClientResponse;
    }

    @Override
    public String getName() {
        return "Langfuse Prompt Tracing Advisor";
    }

    @Override
    public int getOrder() {
        return ToolCallingAdvisor.DEFAULT_ORDER + 100;
    }

    private String serializePrompt(ChatClientRequest request) {
        var messages = request.prompt()
                .getInstructions()
                .stream()
                .map(message -> Map.of(
                        "role", message.getMessageType().getValue(),
                        "content", ofNullable(message.getText()).orElse("")))
                .toList();
        return toJson(messages);
    }

    private String serializeResponse(ChatResponse response) {
        return ofNullable(response).map(ChatResponse::getResult)
                .map(Generation::getOutput)
                .map(output -> toJson(

                        Map.of("role", output.getMessageType().getValue(),
                                "content", ofNullable(output.getText()).orElse(""),
                                "hasToolCalls", response.hasToolCalls())

                ))
                .orElse("");
    }

    private String toJson(Object value) {
        try {
            return this.objectMapper.writeValueAsString(value);
        }
        catch (JsonProcessingException ex) {
            return String.valueOf(value);
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Why getOrder() Is Important

ToolCallingAdvisor manages the tool-calling loop.

If our advisor is inside this loop, it can see intermediate LLM calls:

  1. The user prompt.
  2. The model response that decides to call a tool.
  3. The prompt after the tool result.
  4. The final model response.

For a demo, this is enough to see prompts and responses in Langfuse.

7. Starting Langfuse

Langfuse can be started with Docker Compose:

cp .env.langfuse.example .env.langfuse

docker compose \
  --env-file .env.langfuse \
  -f compose.langfuse.yml \
  up -d
Enter fullscreen mode Exit fullscreen mode

After startup, open:

http://localhost:3000
Enter fullscreen mode Exit fullscreen mode

Create a project and copy:

  • the public key;
  • the secret key.

8. Starting the Application 

export OPENAI_API_KEY=your-openai-key
export LANGFUSE_TRACING_ENABLED=true
export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
export LANGFUSE_PROMPT_TRACING_ENABLED=true

./gradlew bootRun
Enter fullscreen mode Exit fullscreen mode

Test tool calling:

curl "http://localhost:8080/ai?message=What%20is%20the%20weather%20in%20Berlin?"
Enter fullscreen mode Exit fullscreen mode

9. What You Will See in Langfuse

Langfuse should show traces with spans.

Tool calling

Prompt tracing

Generation spans will contain attributes such as:

langfuse.observation.type=generation
langfuse.observation.input=...
langfuse.observation.output=...
input.value=...
output.value=...
Enter fullscreen mode Exit fullscreen mode

For tool calls, Spring AI observability may add separate spans such as:

execute_tool getCurrentWeather
Enter fullscreen mode Exit fullscreen mode

When the following option is enabled:

spring.ai.tools.observations.include-content=true
Enter fullscreen mode Exit fullscreen mode

Spring AI may also export tool-call arguments and results.

10. Production Notes

Prompts, completions, and tool results may contain sensitive information.

Also, traces are usually stored longer than application runtime memory.

For production, consider adding:

  • content truncation;
  • sanitization;
  • secret redaction;
  • PII filtering.

Conclusion

Spring AI provides a convenient extension model through advisors, while Langfuse accepts OTLP spans.

By connecting them, we get clear tracing for:

  • LLM calls;
  • prompts;
  • responses;
  • tool calls;
  • intermediate tool-calling steps.

Top comments (0)