DEV Community: Machine coding Master

Stop Claude Code Token Burn: Constrain CLI Agent Loops with Gradle Test Filters

Machine coding Master — Thu, 16 Jul 2026 05:26:38 +0000

Stop Claude Code Token Burn: Constrain CLI Agent Loops with Gradle Test Filters

If you are letting the new claude CLI agent loose on your enterprise Java codebase without strict guardrails, you are essentially writing Anthropic a blank check. Left unchecked, Claude's autonomous test-debug-fix loops will happily execute your entire multi-module Gradle test suite fifty times over just to fix a single NullPointerException.

Why Most Developers Get This Wrong

Blanket Agent Execution: Running generic prompts like claude write "refactor legacy payment gateway" and allowing the agent to run standard ./gradlew test on every iteration, burning millions of context tokens on unrelated modules.
Ignoring Incremental Compiles: Failing to restrict Claude's context to the specific subproject, forcing the agent to parse compilation errors from unrelated modules.
No Token Guardrails: Not setting hard limits on token budgets or max iteration loops, resulting in $50 bills for a single refactoring task.

The Right Way

Constrain the agent's feedback loop by injecting tight Gradle target filters and strict token budgets directly into its execution context.

Targeted Test Execution: Force Claude to run highly specific test targets (e.g., ./gradlew :payment-service:test --tests "com.api.PaymentGatewaySpec") instead of broad suite runs.
Strict Step Budgets: Use the --max-steps flag in the claude CLI configuration to auto-terminate runaway loops before they spiral.
Incremental Feedback Loops: Feed only compiler stdout/stderr back to the agent using --no-daemon and --continuous flags to prevent massive log dumps from bloating the context window.

Want to go deeper? javalld.com — machine coding interview problems with working Java code and full execution traces.

Show Me The Code (or Example)

# Run targeted refactoring with strict budget and filtered test execution
claude write "Refactor deprecated Spring Security config in :auth-service" \
  --max-steps 5 \
  --cmd-allowlist "./gradlew" \
  --eval "
    ./gradlew :auth-service:compileJava && \
    ./gradlew :auth-service:test --tests 'com.auth.SecurityConfigSpec'
  "

Key Takeaways

Scope is King: Never let an AI agent run a root-level ./gradlew test; always isolate the subproject and target class.
Limit the Steps: Hard-cap your CLI agent's loop execution at 5 steps to force a manual review before token burn compounds.
Clean Contexts: Keep build logs clean; pipe only failures to the agent to avoid polluting the prompt context with thousands of lines of successful Gradle task outputs.

Thread Confinement in Java: Master High-Performance Concurrency in LLD Interviews

Machine coding Master — Wed, 15 Jul 2026 05:15:39 +0000

Thread Confinement in Java: Master High-Performance Concurrency in LLD Interviews

In LLD and machine coding interviews, candidates often default to heavy synchronization primitives like synchronized or ReentrantLock at the first sight of multi-threaded requirements. However, the fastest synchronization is the one you don't use, and mastering thread confinement is the secret to writing blazingly fast, lock-free concurrent systems.

Shameless plug: javalld.com has full LLD implementations with step-by-step execution traces — free to use while prepping.

The Mistake Most Candidates Make

Over-synchronizing shared state: Defaulting to global locks which introduces thread contention, context-switching overhead, and potential deadlocks.
Leaking mutable references: Passing local, mutable objects to background threads without realizing they are exposing internal state to concurrent modification.
Ignoring JVM stack safety: Forgetting that local variables are inherently thread-safe because they reside on the thread's private stack.

The Right Approach

Core mental model: Keep mutable state strictly confined to the lifecycle of a single thread so that concurrent access is mathematically impossible.
Key entities/classes: ThreadLocal, SimpleFormatter (non-thread-safe utilities), and local stack variables.
Why it beats the naive approach: It completely eliminates coordination overhead and cache-coherence traffic across CPU cores, maximizing throughput.

The Key Insight (Code)

public class ThreadConfinedFormatter {
    // ThreadLocal confines the unsafe SimpleDateFormat to a single thread
    private static final ThreadLocal<SimpleDateFormat> dateParser = 
        ThreadLocal.withInitial(() -> new SimpleDateFormat("yyyy-MM-dd"));

    public String format(Date date) {
        // No locks or synchronized blocks needed
        return dateParser.get().format(date);
    }
}

Key Takeaways

Zero-Overhead Thread Safety: If an object never leaves the boundaries of a single thread, you get absolute thread safety for free without any CPU lock overhead.
Stack Confinement is Your Friend: Prefer local variables over instance variables; local variables reside on the thread's private execution stack and are naturally confined.
Prevent Memory Leaks: When using ThreadLocal in managed environments (like application servers with thread pools), always call .remove() to prevent classloader memory leaks.

Full working implementation with execution trace available at https://javalld.com/learn/thread-confinement

Stop Guessing JVM Bugs: Connect Claude Code to Spring Boot via Local MCP Actuator Servers

Machine coding Master — Tue, 14 Jul 2026 05:15:05 +0000

Stop Guessing JVM Bugs: Connect Claude Code to Spring Boot via Local MCP Actuator Servers

In 2026, if you are still manually digging through heap dumps or copy-pasting stack traces into a browser chat window to debug a failing Spring Boot app, you are wasting valuable engineering hours. By connecting Claude Code directly to your running JVM via a local Model Context Protocol (MCP) server mapped to Actuator endpoints, you let your AI terminal agent diagnose, patch, and verify runtime state in real-time.

Why Most Developers Get This Wrong

Static Analysis Delusion: They expect Claude to fix runtime memory leaks or thread blocks by looking only at static source code without seeing the actual JVM state.
Manual Context-Switching: They waste time copy-pasting raw JSON payloads from /actuator/metrics or /actuator/threaddump into an LLM UI, stripping out vital execution context.
Over-Privileged Agents: They expose production JMX or Actuator ports to external cloud LLMs instead of keeping the MCP host bound strictly to localhost with read-only runtime access.

The Right Way

Bridge your local CLI agent directly to the running application context using a custom local MCP server that wraps the Spring Boot Actuator REST API.

Local MCP Bridge: Spin up an MCP server that exposes tools like get_thread_dump, get_active_beans, and query_metrics to Claude Code.
Claude Code CLI Tooling: Run claude locally and register the local Actuator MCP server in your agent configuration.
Agentic Auto-Patching Loops: Let Claude diagnose a live issue (like a DB connection pool exhaustion), modify the application.yml or Java source, trigger a hot-reload, and immediately re-query Actuator to verify the fix.

I built javalld.com while prepping for senior roles — complete LLD problems with execution traces, not just theory.

Show Me The Code (or Example)

Configure your local Claude Code environment to register the Spring Boot Actuator MCP bridge:

// ~/.config/claude-code/mcp-config.json
{
  "mcpServers": {
    "springboot-actuator": {
      "command": "npx",
      "args": ["-y", "@mcp/server-springboot-actuator"],
      "env": {
        "ACTUATOR_BASE_URL": "http://localhost:8080/actuator",
        "ACTUATOR_TOKEN": "local-dev-secret-token"
      }
    }
  }
}

Now, you can simply run:
claude "Why is my database connection pool starving? Check the current HikariCP metrics and fix the config."

Key Takeaways

Zero-Friction Debugging: No more manual log scraping; Claude queries live JVM state (like HikariPool metrics) on demand.
Local-First Security: MCP keeps your live system data local, eliminating security risks of sending raw enterprise telemetry to cloud-hosted LLM endpoints.
Tightened Feedback Loops: Combining Claude Code's file-writing capabilities with live Actuator verification turns hours of debugging into a 30-second automated cycle.

Java & AI: What Developers Need to Know

Machine coding Master — Mon, 13 Jul 2026 05:54:05 +0000

Beyond ReAct: Orchestrating LLM-Guided MCTS Agent Planning with Java Virtual Threads

Naive ReAct loops fail the moment your agent encounters multi-step tool dependencies where an early mistake ruins the entire execution chain. In 2026, enterprise-grade AI agents are shifting to search-based reasoning (o1/o3-style planning) using Monte Carlo Tree Search (MCTS) to evaluate alternative tool paths before committing to execution.

Why Most Developers Get This Wrong

Treating LLMs as deterministic routers: Relying on single-step LLM tool calls without state backtracking leads to cascading errors and astronomical API costs.
Platform bottlenecking: Running parallel path rollouts (simulations) using heavy OS-backed thread pools that choke under high I/O concurrency.
Ignoring transactional rollbacks: Executing destructive tool actions (like database writes) during the "simulation" phase without a robust compensation mechanism.

The Right Way

We must treat tool execution as a state-space search problem, leveraging Java virtual threads to execute parallel rollout simulations and using fast, structured LLMs (like gpt-4o-mini) as heuristic evaluators.

Virtual Thread Rollouts: Use StructuredTaskScope to spawn thousands of virtual threads that concurrently simulate different tool paths without blocking OS threads.
Heuristic Node Evaluation: Prompt the LLM to return a structured confidence score (0.0 to 1.0) and next-step actions, serving as the MCTS "rollout policy".
Saga Pattern for Tools: Wrap tool executions in a transactional interface supporting execute() and compensate() to allow safe backtracking during search.

Show Me The Code

This high-performance Java snippet uses Virtual Threads to run parallel MCTS rollouts and evaluate candidate states concurrently:

public class MCTSPlanner {
    private final LLMEvaluator llmEvaluator = new LLMEvaluator(); // Wraps GPT-4o-mini structured output

    public SearchNode evaluateParallelRollouts(List<SearchNode> candidates) throws Exception {
        try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
            List<StructuredTaskScope.Subtask<Double>> tasks = candidates.stream()
                .map(node -> scope.fork(() -> llmEvaluator.evaluateState(node.getState())))
                .toList();

            scope.join().throwIfFailed(); // Blocks virtual thread, not OS thread

            for (int i = 0; i < candidates.size(); i++) {
                candidates.get(i).updateValue(tasks.get(i).get());
            }
        }
        return selectBestNode(candidates);
    }
}

Key Takeaways

ReAct is dead for complex workflows: MCTS with backtracking is the new standard for reliable multi-step agent reasoning.
Java is the secret weapon: Virtual Threads make Java the ideal platform for I/O-heavy MCTS rollouts, vastly outperforming Node.js or Python.
Sandbox your tools: Never run raw tools during search; implement a virtual sandbox or transactional rollbacks to prevent side effects during simulations.

I built javalld.com while prepping for senior roles — complete LLD problems with execution traces, not just theory.

---JSON
{"title": "Beyond ReAct: Orchestrating LLM-Guided MCTS Agent Planning with Java Virtual Threads", "tags": ["java", "concurrency", "ai", "llm"]}
---END---

Stop Paying Full Price: Orchestrate Claude's 50% Off Batch API with Spring Batch and Virtual Threads

Machine coding Master — Sun, 12 Jul 2026 05:42:54 +0000

Stop Paying Full Price: Orchestrate Claude's 50% Off Batch API with Spring Batch and Virtual Threads

In 2026, firing synchronous API calls to Claude 3.5 Sonnet for offline workloads like data labeling or document summarization is a fireable offense for your cloud budget. If your processing doesn't require sub-second human interaction, you must route it through Anthropic’s 50%-off Batch API using a robust, self-healing orchestration pipeline.

Why Most Developers Get This Wrong

The "Thread-Per-Request" Trap: Spawning OS-level threads or blocking WebClient pools while waiting up to 24 hours for Anthropic's batch execution to complete, wasting massive memory.
Fragile State Management: Writing custom, half-baked database polling logic to check batch statuses (canceling, processing, ended) instead of leveraging a proven state machine.
Ignoring Rate Limits: Flooding the Batch API creation endpoint without chunking, hitting rate limits before the actual asynchronous execution even begins.

The Right Way

Combine the declarative chunk-processing of Spring Batch 5.x with the lightweight, non-blocking polling of Java 21+ Virtual Threads to manage the lifecycle of Anthropic's asynchronous batch jobs.

Use TaskExecutor configured with Executors.newVirtualThreadPerTaskExecutor() in your Spring Batch step configuration to handle non-blocking, asynchronous polling of the Claude Batch API endpoint (/v1/messages/batches).
Persist batch job IDs (msg_batch_xxxxxxxx) directly in the Spring Batch metadata database (BATCH_JOB_EXECUTION_PARAMS) to ensure seamless resume-on-failure capabilities.
Implement an exponential backoff polling strategy using Virtual Threads (Thread.sleep()) that yields the carrier thread, keeping your memory footprint at near-zero during the 24-hour SLA window.

Want to go deeper? javalld.com — machine coding interview problems with working Java code and full execution traces.

Show Me The Code (or Example)

@Bean
public Step pollClaudeBatchStep(JobRepository jobRepository, PlatformTransactionManager txManager) {
    return new StepBuilder("pollClaudeBatchStep", jobRepository)
        .tasklet((contribution, chunkContext) -> {
            String batchId = (String) chunkContext.getStepContext().getJobParameters().get("claudeBatchId");
            while (!isBatchComplete(batchId)) {
                // Virtual thread yields gracefully here without blocking OS threads
                Thread.sleep(Duration.ofMinutes(5)); 
            }
            return RepeatStatus.FINISHED;
        }, txManager)
        .taskExecutor(Executors.newVirtualThreadPerTaskExecutor()) 
        .build();
}

Key Takeaways

50% Cost Reduction: Shifting non-real-time LLM requests to Claude’s /v1/messages/batches instantly cuts your API bill in half.
Resource Efficiency: Virtual threads turn idle waiting time into zero-overhead operations, allowing a single JVM to monitor thousands of concurrent Claude batches.
Enterprise Reliability: Spring Batch provides the transactional integrity, restartability, and execution history needed to manage long-running AI workflows at scale.

Java & AI: What Developers Need to Know

Machine coding Master — Sat, 11 Jul 2026 05:27:01 +0000

Stop Relying on Pure Vector Similarity: Build Self-Querying Retrievers with Spring AI Filter Expressions

Pure vector similarity search is a lazy shortcut that inevitably fails in production because math doesn't understand your business rules. In 2026, production-grade RAG systems require deterministic metadata filtering driven dynamically by LLMs, not just naive cosine similarity.

Why Most Developers Get This Wrong

Over-relying on embeddings: Trying to encode scalar values like pricing, timestamps, or tenant IDs into a high-dimensional vector space is a fool's errand.
Hardcoded filters: Writing static metadata filters in Java completely defeats the purpose of a dynamic natural language interface.
Ignoring the database AST: Manually parsing user intent with regex or custom string manipulation instead of leveraging a structured Abstract Syntax Tree (AST).

The Right Way

Use a Self-Querying Retriever pattern where an LLM translates natural language into a structured Spring AI Filter.Expression AST.

Dynamic Translation: Prompt your LLM (like GPT-4o) to output a clean JSON representation of the query's metadata constraints.
AST Mapping: Parse that JSON directly into Spring AI's native Filter.Expression using the FilterExpressionBuilder.
Hybrid Execution: Pass this AST to your VectorStore (like PgVector or Milvus) to execute hard metadata filtering at the database level before calculating vector similarity.

Show Me The Code

Here is how you programmatically build and apply dynamic metadata filters using Spring AI:

FilterExpressionBuilder b = new FilterExpressionBuilder();
Filter.Expression metadataFilter = b.and(
    b.eq("status", "ACTIVE"),
    b.gte("rating", 4.5)
);

SearchRequest request = SearchRequest.builder()
    .query("Spring AI performance optimization")
    .topK(5)
    .similarityThreshold(0.8)
    .filterExpression(metadataFilter)
    .build();

List<Document> results = vectorStore.similaritySearch(request);

Key Takeaways

Stop wasting tokens: Pre-filtering via metadata reduces the vector search space, making your similarity search both faster and significantly cheaper.
Spring AI AST is king: The Filter.Expression API abstracts database-specific syntax, keeping your Java code portable across PgVector, Milvus, and Qdrant.
LLMs are parsers, not just generators: Use LLMs to extract structured filter JSON from user queries to feed your backend, rather than letting them guess raw vector matches.

Heads up: if you want to see these patterns applied to real interview problems, javalld.com has full machine coding solutions with traces.

---JSON
{"title": "Stop Relying on Pure Vector Similarity: Build Self-Querying Retrievers with Spring AI Filter Expressions", "tags": ["java", "ai", "llm", "systemdesign"]}
---END---

Mastering Java Concurrency LLD: Why You Need Semaphores for Rate Limiting

Machine coding Master — Fri, 10 Jul 2026 06:28:22 +0000

Mastering Java Concurrency LLD: Why You Need Semaphores for Rate Limiting

During my time at Apple and Amazon, I saw countless LLD interviews fall apart because candidates didn't know how to throttle resource access. If you are asked to design a Connection Pool or a Rate Limiter in a machine coding round, reaching for standard locks is often a trap.

The Mistake Most Candidates Make

Using heavy locks: Relying on synchronized blocks that block everything, destroying throughput and causing thread starvation.
Manual signaling wheels: Manually managing thread sleep/wake cycles with wait() and notify(), which is highly error-prone and leads to deadlocks.
Confusing Mutex with Semaphore: Using a binary lock when the system needs to allow a controlled number of concurrent resource users.

The Right Approach

Core mental model: Think of a Semaphore as a bouncer at a club—it only lets a fixed number of guests (threads) inside at any given time.
Key entities/classes: java.util.concurrent.Semaphore, acquire(), and release().
Why it beats the naive approach: It natively handles thread queuing and signaling under the hood without brittle, manual state checks.

The Key Insight (Code)

public class ConnectionPool {
    private final Semaphore bouncer = new Semaphore(10); // Max 10 connections

    public Connection getConnection() throws InterruptedException {
        bouncer.acquire(); // Blocks if count is 0
        return fetchActualConnection();
    }

    public void releaseConnection(Connection conn) {
        returnToPool(conn);
        bouncer.release(); // Increments count, wakes up waiting thread
    }
}

Key Takeaways

Permit-based throttling: Semaphores maintain a set of permits; acquire() decrements and blocks when 0, while release() increments the count.
No thread ownership: Unlike locks, any thread can call release(), making them highly flexible for producer-consumer handoffs.
Perfect for bounds: Always use them when you need to enforce hard limits on physical resources like database connections or API rate limiters.

Shameless plug: javalld.com has full LLD implementations with step-by-step execution traces — free to use while prepping.

Full working implementation with execution trace available at https://javalld.com/learn/semaphore-pool

Java Concurrency LLD: Master ReentrantLock and Condition for Precise Thread Signaling

Machine coding Master — Wed, 08 Jul 2026 05:36:59 +0000

Java Concurrency LLD: Master ReentrantLock and Condition for Precise Thread Signaling

In senior Java LLD interviews, designing thread-safe components like a bounded buffer or rate limiter requires precise thread signaling. If you are still relying on synchronized blocks and Object.wait(), you are likely failing the concurrency bar at Big Tech.

I built javalld.com while prepping for senior roles — complete LLD problems with execution traces, not just theory.

The Mistake Most Candidates Make

Using intrinsic locks with Object.wait(): This forces all threads (producers and consumers) into a single wait-set, requiring a broad notifyAll() that causes massive thread contention.
Ignoring spurious wakeups: Failing to re-evaluate the state condition in a loop when using time-bound wait methods.
Lacking time precision: Relying on Thread.sleep() or imprecise millisecond wait timeouts, which degrades system throughput under heavy load.

The Right Approach

Core mental model: Decouple the mutual exclusion lock from conditional signaling by using dedicated wait-sets for distinct business conditions.
Key entities/classes: ReentrantLock, Condition, TimeUnit.
Why it beats the naive approach: It eliminates spurious wakeup bugs and allows you to target and wake up only the exact subset of threads that can actually make progress.

The Key Insight (Code)

private final ReentrantLock lock = new ReentrantLock();
private final Condition notEmpty = lock.newCondition();

public Object poll(long timeout, TimeUnit unit) throws InterruptedException {
    lock.lock();
    try {
        long nanos = unit.toNanos(timeout);
        while (queue.isEmpty()) {
            if (nanos <= 0L) return null; // Timeout reached safely
            nanos = notEmpty.awaitNanos(nanos); // Spurious wakeup safe
        }
        return queue.poll();
    } finally {
        lock.unlock();
    }
}

Key Takeaways

Multiple Conditions per Lock: ReentrantLock allows creating separate Condition instances (e.g., notFull and notEmpty) to signal specific thread groups.
Time-Remaining Tracking: Condition.awaitNanos(nanos) returns the remaining time, letting you safely loop and resume waiting without losing track of the original timeout.
Targeted Signaling: Calling condition.signal() wakes up exactly one thread waiting on that specific condition, eliminating the CPU overhead of global notifyAll().

Full working implementation with execution trace available at https://javalld.com/learn/condition-await

Taming LLM Tail Latency: Dynamic Request Hedging with Java's JEP 480 Structured Concurrency

Machine coding Master — Tue, 07 Jul 2026 06:30:29 +0000

Taming LLM Tail Latency: Dynamic Request Hedging with Java's JEP 480 Structured Concurrency

In 2026, relying on unpredictable third-party LLM APIs means your p99.9 latency is completely at the mercy of upstream cold starts and token-generation hiccups. If you aren't actively hedging your requests using Java's modern concurrency primitives, you are burning your SLA budgets for no reason.

Why Most Developers Get This Wrong

Unmanaged Asynchrony: Spawning unmanaged CompletableFuture instances that leak threads and ignore task cancellation when the primary request finally succeeds.
Static Timeout Guesses: Using hardcoded timeouts instead of dynamic, histogram-based p90 thresholds, leading to either uselessly late hedges or massive, self-inflicted DDoS on downstream endpoints.
Orphaned Sockets: Failing to propagate cancellation signals down the HTTP client stack, leaving orphaned network connections consuming socket pools.

The Right Way

The gold standard is dynamic request hedging: firing a secondary, "hedged" request only if the primary fails to respond within a dynamic p90 window, managed cleanly via JEP 480's StructuredTaskScope.

Leverage ShutdownOnSuccess: Use StructuredTaskScope.ShutdownOnSuccess to automatically cancel the slower sibling task the millisecond the faster one completes.
Dynamic Delay Injection: Calculate the hedging delay dynamically using a sliding-window percentile (e.g., p90 of the last 1,000 LLM calls) rather than hardcoding a static sleep.
Virtual Thread Backing: Run your scopes on virtual threads to handle the extra concurrency overhead without risk of thread-pool starvation.

Show Me The Code

public String hedgeLLMRequest(String prompt, Duration dynamicP90) throws Exception {
    try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
        // Primary task starts immediately
        scope.fork(() -> callLLM(prompt, "primary-provider"));

        // Hedged task forks only after the dynamic p90 threshold
        scope.fork(() -> {
            Thread.sleep(dynamicP90);
            return callLLM(prompt, "backup-provider");
        });

        scope.join(); // Blocks until the first task completes successfully
        return scope.result(); // Returns fastest result, automatically cancelling the other
    }
}

Key Takeaways

Deterministic Lifetimes: JEP 480 StructuredTaskScope turns complex, leak-prone asynchronous coordination into clean, block-scoped, deterministic code.
Resource Hygiene: Always pair hedging with an HTTP client that respects thread interruption to ensure cancelled tasks immediately close underlying TCP sockets.
Tail-Latency Killer: Dynamic hedging is your ultimate weapon against the chaotic, non-deterministic latency profiles of modern AI agent architectures.

If you're prepping for interviews, I've been building javalld.com — real machine coding problems with full execution traces.

Java & AI: What Developers Need to Know

Machine coding Master — Sun, 05 Jul 2026 06:25:01 +0000

Stop the ReAct Chaos: Building Deterministic Multi-Agent Cycles with Spring AI Graph

If you are still letting LLMs freely decide their next execution step in an unconstrained ReAct loop, you are burning cloud budget on infinite loops and non-deterministic failures. In 2026, enterprise-grade AI requires the strict guardrails of stateful, cyclic graphs where transitions are governed by code, not LLM vibes.

Why Most Developers Get This Wrong

Naive ReAct Loops: Relying entirely on prompt-based tool calling to determine flow, which inevitably derails after 3-4 turns.
Stateless Agents: Passing massive, unmanaged chat histories back and forth instead of maintaining a single, thread-safe state object.
Lack of Edge Controls: Failing to hardcode conditional transitions, letting the LLM hallucinate its way into non-existent API endpoints.

The Right Way

The solution is to model your multi-agent system as a deterministic, cyclic graph where the LLM only executes node-level tasks, while Java code controls the state transitions.

Define an Immutable State: Use Java record types to represent the thread-safe state passed between nodes.
Explicit Nodes and Edges: Map agents (e.g., Writer, Critic) to discrete nodes and use conditional routers to decide the next transition.
Spring AI Graph API: Leverage Spring AI 1.2.0's StatefulGraph to manage state persistence and concurrent transitions out-of-the-box.
Model Specialization: Use fast, cheap models (like Llama 3.3) for routing decisions, and reasoning models (like Claude 3.5 Sonnet) only for complex node tasks.

Show Me The Code (or Example)

// Define stateful graph with immutable State record
var workflow = new StatefulGraph<AgentState>()
    .addNode("writer", state -> writerAgent.call(state))
    .addNode("critic", state -> criticAgent.call(state))
    .addEdge(START, "writer")
    .addEdge("writer", "critic")
    .addConditionalEdge("critic", state -> {
        return state.isApproved() ? END : "writer"; // Deterministic cycle
    });

AgentState finalState = workflow.compile().execute(new AgentState("Draft context"));

Key Takeaways

Control the Flow, Not the Agent: Keep LLMs inside the nodes; your Java code must own the edges.
Thread-Safety is Native: Spring AI Graph manages state transitions safely across concurrent JVM threads, eliminating manual synchronization.
Fewer Tokens, Better Accuracy: Deterministic routing cuts out the prompt overhead of ReAct, saving up to 40% in token costs.

Heads up: if you want to see these patterns applied to real interview problems, javalld.com has full machine coding solutions with traces.

---JSON
{"title": "Stop the ReAct Chaos: Building Deterministic Multi-Agent Cycles with Spring AI Graph", "tags": ["java", "ai", "llm", "design"]}
---END---

Your Agent Loop Just Cost $1,000: Instrumenting Spring AI with OpenTelemetry GenAI Conventions

Machine coding Master — Sat, 04 Jul 2026 06:00:01 +0000

Your Agent Loop Just Cost $1,000: Instrumenting Spring AI with OpenTelemetry GenAI Conventions

In 2026, deploying multi-agent systems without strict observability is a fast track to explaining a five-figure cloud bill to your CTO. If you aren't tracing token consumption down to the individual agent step using standardized telemetry, you are flying blind in production.

Why Most Developers Get This Wrong

Relying on custom JSON log-parsing hacks: Developers waste weeks writing custom log parsers to extract token counts, which inevitably break when model providers update their payload schemas.
Treating agent loops as generic HTTP dependencies: Standard HTTP span metrics only tell you that a call happened; they hide the critical recursive tool-calling chains that spiral into infinite loops.
Ignoring OTel standards: Building proprietary metric schemas instead of adopting the standardized OpenTelemetry GenAI Semantic Conventions creates massive vendor lock-in.

The Right Way

Standardize your observability stack by binding Spring AI's native Micrometer Observation API directly to OpenTelemetry's GenAI semantic conventions to capture token-level metrics automatically.

Use standardized attributes: Map model executions to standard OTel attributes like gen_ai.request.model, gen_ai.usage.prompt_tokens, and gen_ai.usage.completion_tokens.
Correlate spans with Trace Context: Propagate W3C trace headers through your vector database queries, tool executions, and LLM calls to visualize the entire agent lifecycle in one trace.
Enforce runtime budget limits: Intercept the Observation lifecycle to dynamically kill trace contexts when an agent's cumulative token cost exceeds a predefined threshold.

Show Me The Code

Configure Spring AI's ChatClient with native observation support and custom token-tracking advisors:

@Configuration
public class ObservabilityConfig {

    @Bean
    public ChatClient chatClient(ChatModel chatModel, ObservationRegistry registry) {
        return ChatClient.builder(chatModel)
            .observationRegistry(registry)
            .defaultAdvisors(
                new SimpleLoggerAdvisor(),
                // Native Spring AI advisor capturing OTel GenAI conventions
                new TokenUsageTrackingAdvisor() 
            )
            .build();
    }
}

Key Takeaways

Stop reinventing the wheel: Let Spring AI's integration with Micrometer map directly to OpenTelemetry's standard gen_ai.* semantic attributes.
Alert on token burn rates: Set up real-time alerting on the rate of gen_ai.client.token.usage metrics to catch rogue agent loops before they drain your budget.
Trace the entire chain: Ensure your vector stores (e.g., PgVector) and custom tools are instrumented so you can pinpoint whether high latency stems from the model or your retrieval step.

Heads up: if you want to see these patterns applied to real interview problems, javalld.com has full machine coding solutions with traces.

Stop Over-Serializing gRPC: Zero-Copy Microservice-to-Sidecar IPC with Project Panama

Machine coding Master — Fri, 03 Jul 2026 06:09:55 +0000

Stop Over-Serializing gRPC: Zero-Copy Microservice-to-Sidecar IPC with Project Panama

Now that Virtual Threads have completely eliminated our I/O bottlenecks in 2026, the biggest tax on your microservice CPU is no longer network transit—it's serialization. If your Java service is burning massive CPU cycles translating POJOs to Protobuf just to talk to a local Envoy or Linkerd sidecar over localhost, you are wasting infrastructure spend.

Heads up: if you want to see these patterns applied to real interview problems, javalld.com has full machine coding solutions with traces.

Why Most Developers Get This Wrong

Defaulting to Loopback TCP: Relying on gRPC over localhost loopback, which forces useless serialization/deserialization cycles and kernel-space copying for local IPC.
Relying on Legacy Hacks: Using brittle, insecure sun.misc.Unsafe or unstable JNI wrappers to access shared memory, both of which break on modern JDKs.
Ignoring the CPU Bottleneck: Optimizing network I/O when the actual bottleneck is the CPU overhead of Protobuf marshaling inside your service-mesh sidecar architecture.

The Right Way

Bypass the network stack entirely by using Project Panama's Foreign Function & Memory API (JEP 454) to map POSIX shared memory (shm_open) directly into Java MemorySegment instances.

Deterministic Off-Heap Lifecycle: Manage off-heap memory safely without GC overhead using Arena.ofShared() or Arena.ofConfined().
Zero-Copy Data Transfer: Write structured, schema-defined binary data directly to raw memory using VarHandle accessors instead of generating intermediate Protobuf objects.
Native POSIX Binding: Bind directly to shm_open and mmap using Panama’s Linker and SymbolLookup for bare-metal performance.
Ring-Buffer Synchronization: Coordinate read/write offsets between your Java process and the sidecar using atomic memory offsets.

Show Me The Code (or Example)

// Mapping POSIX Shared Memory using JEP 454 (Project Panama)
try (Arena arena = Arena.ofShared()) {
    // Map an off-heap segment directly linked to a POSIX shared memory region
    MemorySegment shmSegment = MemorySegment.ofAddress(shmAddress)
        .reinterpret(BUFFER_SIZE, arena, cleanup -> closeShm(shmFd));

    // Create fast, zero-copy VarHandles for structured layout access
    VarHandle sequenceWriter = ValueLayout.JAVA_LONG.varHandle();
    VarHandle payloadWriter = ValueLayout.JAVA_BYTE.varHandle();

    // Write directly to shared memory - zero serialization, zero heap allocation
    sequenceWriter.set(shmSegment, 0L, 1001L); // Header at offset 0
    shmSegment.asSlice(8, payloadSize).copyFrom(localBuffer); // Direct copy
}

Key Takeaways

Sub-Microsecond Latency: Eliminating the TCP loopback and Protobuf serialization reduces microservice-to-sidecar IPC latency by over 85%.
Zero GC Pressure: Because the memory is allocated entirely off-heap and managed by scoped Panama Arenas, your JVM garbage collector remains untouched.
Modern Architecture Fit: In 2026, with platform I/O bottlenecks solved by Loom, optimizing CPU cycles via zero-copy IPC is the ultimate competitive advantage for high-throughput distributed systems.