GitHub's Copilot SDK for Java: What Running Agents in Spring Boot Without Frameworks Reveals About Tool Integration

Every Java team adding AI to a backend right now faces the same fork in the road. If you are on Spring Boot, you reach for Spring AI. If you are not, you reach for LangChain4j. Both are good libraries, but both come with a commitment: you adopt their abstractions, their release cadence, and their opinion of what an agent loop looks like.

On August 10, 2026, GitHub quietly published a third option. The Copilot SDK for Java embeds the same agent runtime that powers Copilot CLI as a Maven dependency you can drop into any server-side Java application. It runs against OpenAI, Anthropic, Azure, or any OpenAI-compatible endpoint with your own API key. No Copilot subscription required.

This is not a thin wrapper over a model API. It is a production-tested agent runtime with tool calling, streaming, and context management built in. The interesting part is what it exposes about the minimal plumbing needed to run agents in a servlet container without framework overhead.

Full disclosure: this evaluation is based on GitHub's official documentation, the SDK README, and engineering posts from the release team. I have not shipped the Copilot SDK in production yet.

What the SDK Actually Is

The Copilot SDK is not a model client. It is an agent runtime. The distinction matters.

Most Java AI libraries give you a thin wrapper over a provider's HTTP endpoint: send messages, get a completion. The Copilot SDK exposes something bigger. It embeds the same engine behind Copilot CLI, which means you get:

• Tool calling with automatic schema generation from Java method signatures
• Streaming responses with backpressure handling for servlet containers
• Context window management that tracks token budgets across multi-turn conversations
• Multi-model routing that lets you switch between GPT-4, Claude, or local models without changing application code

The SDK ships in two modes. The first is GitHub-hosted, where the agent runtime runs in GitHub's cloud and you authenticate with a Copilot subscription. The second is BYOK (bring your own key), where the runtime runs in your JVM and you point it at any OpenAI-compatible endpoint with your own API key.

For production Spring Boot services, BYOK mode is the one that matters. It means the agent loop runs in your process, not GitHub's. You control rate limits, observability, and failure modes.

Integration Surface: What You Actually Wire

Here is what the minimal integration looks like in a Spring Boot controller:

@RestController
@RequestMapping("/api/agent")
public class AgentController {

    private final CopilotClient client;

    public AgentController() {
        this.client = CopilotClient.builder()
            .apiKey(System.getenv("OPENAI_API_KEY"))
            .model("gpt-4")
            .build();
    }

    @PostMapping("/chat")
    public Flux<String> chat(@RequestBody ChatRequest request) {
        AgentSession session = client.createSession();

        // Register tools as Java methods
        session.registerTool("getWeather", this::getWeather);
        session.registerTool("searchDocs", this::searchDocs);

        // Stream response with automatic tool calling
        return session.sendMessage(request.getMessage())
            .map(chunk -> chunk.getContent());
    }

    private String getWeather(String location) {
        // Tool implementation
        return weatherService.fetch(location);
    }

    private String searchDocs(String query) {
        // Tool implementation
        return docService.search(query);
    }
}

The SDK handles tool schema generation automatically. You do not write JSON schemas by hand. You do not annotate methods with @Tool like LangChain4j. You pass a method reference, and the SDK introspects the signature to build the function definition that gets sent to the model.

This is the first major difference from Spring AI and LangChain4j. Both frameworks require you to define tools through their abstraction layers. Spring AI uses @Bean definitions with FunctionCallback. LangChain4j uses @Tool annotations on service methods. The Copilot SDK just takes a lambda or method reference.

Tool Calling: Schema Generation and Execution Flow

The SDK's tool-calling flow has three steps:

1. Registration: You pass a method reference to session.registerTool(). The SDK uses reflection to extract parameter types, names, and return type.
2. Schema generation: The SDK builds an OpenAI function definition from the method signature. A method like String getWeather(String location, boolean includeHumidity) becomes a JSON schema with two parameters.
3. Execution: When the model returns a function call, the SDK invokes your method with the arguments the model provided, then sends the result back in the next turn.

This happens automatically. You do not write a tool executor. You do not handle the function call response format. The SDK manages the loop.

Here is what that looks like under the hood:

// SDK generates this schema from your method signature
{
  "name": "getWeather",
  "description": "Fetches current weather for a location",
  "parameters": {
    "type": "object",
    "properties": {
      "location": { "type": "string" },
      "includeHumidity": { "type": "boolean" }
    },
    "required": ["location"]
  }
}

The SDK sends this schema to the model in the tools array. When the model decides to call the function, the SDK parses the arguments, invokes your method, and appends the result to the conversation history.

The execution flow is synchronous by default. If your tool method blocks (database query, HTTP call), the agent loop blocks. The SDK does not provide async tool execution out of the box. You need to handle that in your method implementation, either by returning a CompletableFuture or using Spring's @Async.

Streaming and Backpressure in Servlet Containers

The SDK returns streaming responses as a Publisher<ChatChunk> (Reactive Streams). In Spring Boot, you map this to a Flux<String> and return it from a controller method. Spring WebFlux handles the backpressure.

This is the second major difference from Spring AI. Spring AI's streaming API returns a Flux<ChatResponse>, but it wraps the underlying provider's SSE stream in a Spring-specific abstraction. The Copilot SDK gives you raw Reactive Streams, which means you can plug it into any reactive runtime (Reactor, RxJava, Mutiny) without framework coupling.

The backpressure handling is important in servlet containers. If the client is slow to consume chunks, the SDK pauses the upstream request to the model API. This prevents memory buildup in the JVM. Spring AI does the same thing, but LangChain4j's streaming API does not expose backpressure controls by default. You get an Iterator<String>, and if you do not consume it fast enough, chunks buffer in memory.

Here is what the streaming flow looks like:

@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ServerSentEvent<String>> streamChat(@RequestParam String message) {
    AgentSession session = client.createSession();

    return session.sendMessage(message)
        .map(chunk -> ServerSentEvent.builder(chunk.getContent()).build())
        .doOnError(e -> log.error("Stream error", e))
        .doFinally(signal -> session.close());
}

The SDK handles token counting internally. Each ChatChunk includes a getTokenCount() method that returns the cumulative token usage for the session. You can use this to enforce budget limits or log usage for billing.

Context Window Management and Session State

The SDK manages conversation history automatically. When you create an AgentSession, it tracks all messages (user, assistant, tool calls, tool results) in memory. You do not manually append to a message list.

This is convenient for prototyping, but it creates a problem in production: sessions grow unbounded. If you run a long conversation, the context window fills up, and the SDK throws a ContextWindowExceededException.

The SDK does not provide automatic truncation or summarization. You need to handle this yourself. The session API exposes a getMessages() method that returns the full history, and a setMessages() method that lets you replace it. You can implement a sliding window, summarize old turns, or drop tool calls that are no longer relevant.

Here is a simple sliding window implementation:

public class BoundedSession {
    private final AgentSession session;
    private final int maxMessages;

    public Flux<ChatChunk> sendMessage(String message) {
        List<Message> history = session.getMessages();

        if (history.size() > maxMessages) {
            // Keep system message and last N turns
            List<Message> truncated = new ArrayList<>();
            truncated.add(history.get(0)); // system message
            truncated.addAll(history.subList(history.size() - maxMessages, history.size()));
            session.setMessages(truncated);
        }

        return session.sendMessage(message);
    }
}

Spring AI provides a MessageHistoryAdvisor that does this automatically. LangChain4j has a ChatMemory abstraction with built-in window strategies. The Copilot SDK gives you the primitives, but you write the policy.

Authentication and Rate Limiting Boundaries

In BYOK mode, the SDK makes direct HTTP calls to the model provider's API. You pass an API key in the client builder, and the SDK includes it in the Authorization header for every request.

This means rate limiting happens at the provider level, not in the SDK. If you hit OpenAI's rate limit, you get a 429 response, and the SDK throws a RateLimitException. You need to handle retries yourself.

The SDK does not provide a built-in retry policy. You can wrap the session in a retry decorator using Resilience4j or Spring Retry:

@Bean
public RetryTemplate retryTemplate() {
    return RetryTemplate.builder()
        .maxAttempts(3)
        .exponentialBackoff(1000, 2, 10000)
        .retryOn(RateLimitException.class)
        .build();
}

@Service
public class ResilientAgentService {
    private final CopilotClient client;
    private final RetryTemplate retryTemplate;

    public Flux<String> chat(String message) {
        return retryTemplate.execute(ctx -> 
            client.createSession().sendMessage(message)
                .map(chunk -> chunk.getContent())
        );
    }
}

Spring AI includes a RateLimiter advisor that handles this automatically. LangChain4j does not. The Copilot SDK is closer to LangChain4j in this regard: you get the error, you handle the retry.

Observability: What You Can and Cannot See

The SDK does not emit structured logs or metrics by default. You get Java logging at the DEBUG level, which includes HTTP request/response bodies and token counts. That is it.

If you want distributed tracing, you need to instrument the session manually. The SDK does not integrate with OpenTelemetry or Micrometer out of the box. You can wrap session methods in spans:

@Service
public class ObservableAgentService {
    private final CopilotClient client;
    private final Tracer tracer;

    public Flux<String> chat(String message) {
        Span span = tracer.spanBuilder("agent.chat").startSpan();

        return client.createSession().sendMessage(message)
            .map(chunk -> {
                span.addEvent("chunk.received", 
                    Attributes.of(AttributeKey.longKey("tokens"), chunk.getTokenCount()));
                return chunk.getContent();
            })
            .doFinally(signal -> span.end());
    }
}

Spring AI provides a ChatClientObservationConvention that emits Micrometer metrics and OpenTelemetry spans automatically. LangChain4j has a ChatModelListener interface you can implement to hook into the request/response cycle. The Copilot SDK does not have an equivalent. You instrument it like any other HTTP client.

Model Selection and Configuration

The SDK exposes model selection, temperature, and token budgets through the client builder:

CopilotClient client = CopilotClient.builder()
    .apiKey(System.getenv("OPENAI_API_KEY"))
    .model("gpt-4")
    .temperature(0.7)
    .maxTokens(2000)
    .build();

This is static configuration. You set it when you build the client, and it applies to all sessions created from that client. If you need per-request model selection, you need to create multiple clients or rebuild the client on each request.

Spring AI uses Spring's configuration system (application.yml) and supports property placeholders and profiles. LangChain4j uses builder patterns like the Copilot SDK. The Copilot SDK does not integrate with Spring's @ConfigurationProperties, so you need to wire it manually:

@Configuration
public class CopilotConfig {

    @Value("${copilot.api-key}")
    private String apiKey;

    @Value("${copilot.model}")
    private String model;

    @Bean
    public CopilotClient copilotClient() {
        return CopilotClient.builder()
            .apiKey(apiKey)
            .model(model)
            .build();
    }
}

Comparison: SDK vs. Spring AI vs. LangChain4j

Feature	Copilot SDK	Spring AI	LangChain4j
Tool schema generation	Automatic from method signature	Manual FunctionCallback beans	@Tool annotations
Streaming backpressure	Reactive Streams Publisher	Spring Flux wrapper	Iterator (no backpressure)
Context window management	Manual truncation via setMessages()	MessageHistoryAdvisor	ChatMemory strategies
Retry and rate limiting	Manual (Resilience4j, Spring Retry)	Built-in RateLimiter advisor	Manual
Observability	Manual instrumentation	Micrometer + OpenTelemetry	ChatModelListener hooks
Configuration	Builder pattern	Spring @ConfigurationProperties	Builder pattern
Framework coupling	None (works with any servlet container)	Requires Spring Boot	None

The Copilot SDK is the leanest option. It gives you the agent runtime and nothing else. Spring AI is the most integrated option. It hooks into Spring's configuration, observability, and retry infrastructure. LangChain4j sits in the middle: it provides abstractions for tools and memory, but it does not integrate with Spring's ecosystem.

Deployment Shape and Failure Modes

In BYOK mode, the SDK runs entirely in your JVM. There is no external orchestration service. The agent loop is a synchronous call stack: your controller