I wanted to actually learn LangChain4j and LangGraph4j beyond the "hello world" chat example, so I built something with enough moving parts to force real decisions: a hiring workflow where several AI agents independently score a candidate, a graph aggregates their opinions into a routing decision, and a human gets pulled in whenever the signals aren't clean. It's langchain4j-sample on Spring Boot, with LangGraph4j's Postgres checkpointer for state.
This post is two things: first, how the framework pieces fit together and what I learned exploring them; second — because I didn't stop at "works against a mock LLM" — what happened when I pointed the same graph at a real, local, CPU-only Ollama model, and what that taught me about running agentic workflows outside a demo.
Why LangGraph4j instead of just chaining LangChain4j calls
The workflow needed three things a plain sequence of LLM calls doesn't give you for free: independent agents that run at the same time, a pause point that waits for a human and resumes later (possibly minutes or days later, possibly after a restart), and a durable record of exactly what happened at each step. That's what pushed me toward LangGraph4j's StateGraph rather than just wiring LangChain4j calls together by hand.
The graph itself reads close to how I'd describe the process out loud:
parseResume
│
├─ unreadable? ──────────────► humanReview
│
▼ (fan out, 4 concurrent agents)
skills · experience · cultureFit · redFlags
│
▼
aggregateScores ──► automated? ──► finalDecision ──► notification
│
└─ not automated ──► humanReview ──(approve/reject/re-analyze)──► finalDecision
ParseResumeNode extracts structured data from raw resume text. If it comes back unreadable, there's no point running four scoring agents against nothing, so the graph short-circuits straight to human review. Otherwise it fans out to four independent agents — skills, experience, culture fit, red flags — each reasoning about a different dimension of fit. AggregateScoresNode combines their verdicts with a weighted formula, and anything that isn't a clean auto-advance or auto-reject drops into a humanReview interrupt. A human can approve, reject, or ask for a targeted re-analysis, which loops back into review until someone makes a final call.
What AiServices made pleasant
Each agent is a plain Java interface — no client boilerplate, no manual JSON parsing:
public interface SkillsAgent {
@UserMessage("""
Candidate ID: {{candidateId}}
Job Requisition ID: {{requisitionId}}
Use the available tools to fetch the job requisition and the candidate. Assess how
well the candidate's skills (from their resume text) match the requisition's required
skills. Score 0-100. Flag any required skill that's missing (e.g.
"MISSING_REQUIRED_SKILL:Kubernetes"). Recommend ADVANCE, CAP, or REJECT.
""")
AgentScoreResult analyzeSkills(@V("candidateId") long candidateId, @V("requisitionId") long requisitionId);
}
AgentScoreResult is a plain record — score, flags, recommendation — and LangChain4j handles getting the model to fill it in and parsing the result back into that shape. Wiring in tools was just as low-friction: AiServices.builder(SkillsAgent.class).chatModel(...).tools(jobRequisitionTool, candidateTool).build(), and the model decides for itself when to call fetchJobRequisition or fetchCandidate mid-conversation. No manual function-calling loop to write.
The part I liked most: swapping LLM backends is pure config, not code. One ChatModel bean is conditional on llm.provider=mock and returns fixture data (great for tests and for exploring the graph's routing logic without burning tokens); another is conditional on llm.provider=ollama and builds a real OllamaChatModel. Every agent is wired against whichever bean is active — the agent interfaces and the graph don't know or care which one it is.
What LangGraph4j made pleasant — and one thing that wasn't obvious
The four scoring agents run through a ParallelNode — you point several edges out of the same source node and LangGraph4j fans them out concurrently. What wasn't obvious from the docs: registering an executor once at graph-compile time isn't enough. Every single invocation needs the executor attached to its own RunnableConfig, or the "parallel" branches just run one after another:
private RunnableConfig runnableConfig(String threadId) {
return RunnableConfig.builder()
.threadId(threadId)
.addParallelNodeExecutor(HiringWorkflowGraph.ANALYSIS_FAN_OUT, analysisFanOutExecutor)
.build();
}
The other genuinely nice piece is interruptBefore(HUMAN_REVIEW) paired with a PostgresSaver checkpointer. The graph pauses inside an in-progress execution, persists its exact state, and returns control to the HTTP request — then, whenever a human decision comes in (hours or days later, possibly against a different app instance after a restart), compiledGraph.invoke(GraphInput.resume(...), sameThreadId) picks the graph back up exactly where it left off. I didn't have to build any of that resumability myself.
One thing that cost me some trial and error: checkpointing serializes state to plain JSON with no type tagging, so a value read back after a reload comes back as a generic Map/String, not the original record. Every accessor on my WorkflowState normalizes through ObjectMapper.convertValue so callers get a consistently-typed result whether the value just came from the same in-memory node-to-node handoff or from a checkpoint reload:
private <T> Optional<T> convert(String key, Class<T> type) {
return this.<Object>value(key)
.map(raw -> type.isInstance(raw) ? type.cast(raw) : OBJECT_MAPPER.convertValue(raw, type));
}
With that in place, and the mock provider driving deterministic fixture data, the whole graph — fan-out, aggregation, human-review interrupt, resume, final decision — worked exactly as designed. That's usually where a learning project like this stops. I kept going, because "works against canned fixtures" and "works against a real model" turned out to be very different claims.
Then I pointed it at a real, local, CPU-only model
I didn't want a cloud API key for a side project, so I ran Ollama locally on an old CPU-only laptop. This is where the framework knowledge from above collided with the actual, physical limits of local inference — and where I learned the most.
Lesson 1: a synchronous call and a busy backend interact in a non-obvious way
My smoke-test script submits eight candidate/requisition pairs with a 900-second timeout per case — generous, I assumed, for a 3B model. Case 1 blew straight through it with no response.
The instinct is to suspect the model is stuck. The real interaction was between two things I'd built independently and never thought to consider together: WorkflowController.submit runs the entire graph invocation synchronously on the request thread (compiledGraph.invoke(...), no timeout of its own), and ollama serve runs with a single execution slot by default (-np 1) — so the four "concurrent" fan-out agents don't run concurrently against it at all, they queue and execute one at a time. When curl gave up at 900 seconds, nothing told the server thread to stop; it kept running, kept holding its place in Ollama's queue, for a client that had already left. Every subsequent request just queued up behind it, since nothing had cancelled it.
The fix taught me something reusable well beyond this project: run the invocation on its own executor with a bounded Future.get(timeout), and call future.cancel(true) on timeout. For a thread blocked inside a JDK HttpClient.send(), that interrupt actually propagates and releases the connection — it's not just a client-side illusion of giving up.
private Optional<WorkflowState> invokeWithTimeout(String threadId, GraphInvocation invocation) throws Exception {
Future<Optional<WorkflowState>> future = workflowInvocationExecutor.submit(invocation::invoke);
try {
return future.get(runTimeoutSeconds, TimeUnit.SECONDS);
} catch (TimeoutException e) {
future.cancel(true);
throw new ResponseStatusException(HttpStatus.GATEWAY_TIMEOUT, ...);
}
}
The generalizable lesson: "parallel" describes your code's intent, not a guarantee about the backend underneath it. Fan four calls out against something with one execution slot, and you've built a queue with extra latency dressed up as concurrency — and if nothing can cancel a request once its caller stops waiting, that queue only grows.
Lesson 2: a "thinking" model taxes you even when you didn't ask it to think
I benchmarked the model I'd configured — qwen3.5:4b — with the simplest possible prompt: "Reply with exactly: OK." It took 35.2 seconds. The response carried a 158-token internal reasoning trace ahead of the two-word answer — this is a reasoning model, and it pays that cost on every call, regardless of how trivial the task is. Switching to qwen2.5:3b, a similarly-sized non-reasoning model I already had pulled, the identical prompt came back in 7.4 seconds with no reasoning overhead at all.
For a real resume-scoring prompt the gap held: qwen2.5:3b answered in 14.9 seconds with 56 tokens of correctly-shaped JSON. Out of curiosity I tried phi3:mini too — 22.9 seconds, 149 tokens, and the JSON was invalid: wrapped in markdown fences despite being told not to, with fields that weren't even in the schema I'd asked for.
What I took from this: for a task with a strict output contract, a model's willingness to reason out loud isn't a bonus, it's overhead you're paying for on every single call whether you wanted it or not. Model choice mattered more here than any amount of prompt tuning afterward.
Lesson 3: an agent that can call tools can also just... not
Once the faster model was in, I re-ran the smoke test and got something odder: every one of eight very different candidates came back with an identical, critical, human-review-forcing flag — POLICY_VIOLATION. Including candidates whose resumes had nothing resembling a policy issue.
I turned on LangChain4j's request/response logging and timestamp-correlated every call in one run to see exactly what each agent actually did. Three of the four scoring agents called their tools (fetchCandidate, fetchJobRequisition, fetchCompanyPolicies) before answering. The red-flags agent's very first response — before any tool call — was already its final answer:
{"score": 75, "redFlags": [{"code": "POLICY_VIOLATION", "severity": "CRITICAL", "forcesHumanReview": true}]}
It never looked at any data. Comparing prompts explained why this agent specifically: its example code, "POLICY_VIOLATION", is a complete, plausible-sounding answer with nothing candidate-specific to fill in. The skills agent's example, "MISSING_REQUIRED_SKILL:Kubernetes", isn't copyable the same way — the model has to supply a real, specific skill name for it to make sense. A model that's uncertain about tool use can lazily echo the first kind of example wholesale; it can't do that with the second.
The fix was making the tool-use requirement explicit and making the example un-copyable:
- Use the available tools to fetch the job requisition, the candidate, and the
- company policies applicable to the requisition's department. ... give a short code
- (e.g. "EMPLOYMENT_GAP", "POLICY_VIOLATION")
+ You must call fetchJobRequisition, fetchCandidate, and fetchCompanyPolicies before
+ forming any opinion — never answer from the candidate ID and requisition ID alone.
+ ... give a short code naming the specific policy or issue (e.g. "EMPLOYMENT_GAP",
+ "POLICY_VIOLATION:NON_COMPETE"). If the fetched data shows no actual gap or policy
+ conflict, return an empty redFlags array — never report a flag you can't tie to
+ something the tools actually returned.
The fabricated critical violations disappeared. Being honest about the rest of the story: re-tracing calls afterward, the model still occasionally skipped tool-calling on its first turn — it just now defaults to a safe {"score": 0, "redFlags": []} instead of a dangerous fabricated one. The prompt change narrowed the blast radius of the underlying behavior; it didn't eliminate it. That's a genuinely useful thing to learn about steering a small model through instructions alone: it has a ceiling.
Lesson 4: knowing where that ceiling actually is
The obvious next step past "ask nicely" is "force it." LangChain4j has exactly the primitive for that:
OllamaChatModel.builder()
.defaultRequestParameters(
ChatRequestParameters.builder().toolChoice(ToolChoice.REQUIRED).build())
.build();
This didn't get silently ignored — it failed fast, at startup, before ever reaching Ollama:
UnsupportedFeatureException: ToolChoice.REQUIRED is not supported yet by this model provider
LangChain4j's own client-side validation refuses to construct the model. As of the version I'm on (1.18.0), forced tool choice just isn't wired up for the Ollama integration yet. Useful to know precisely, rather than spending an afternoon guessing why a "fix" wasn't taking effect: if I want that guarantee today, the actual answer is architectural, not a config flag — fetch the data in Java myself and hand it to the model directly, rather than asking the model to decide whether to fetch it.
Shipping it without asking anyone to install Ollama by hand
None of the above is worth much if trying the project means manually pulling a model before anything works. The Compose setup ended up as three long-running services plus one one-shot:
ollama:
image: ollama/ollama
healthcheck:
test: ["CMD", "ollama", "list"]
interval: 5s
timeout: 5s
retries: 10
# One-shot: pulls the model into ollama's own volume, cached across restarts,
# so the app never races a still-downloading model on first startup.
ollama-pull:
image: ollama/ollama
environment:
OLLAMA_HOST: ollama:11434
entrypoint: ["ollama", "pull", "qwen2.5:3b"]
depends_on:
ollama:
condition: service_healthy
app:
build: .
environment:
LLM_PROVIDER: ollama
OLLAMA_MODEL_NAME: qwen2.5:3b
depends_on:
ollama-pull:
condition: service_completed_successfully
ollama-pull is a throwaway container whose only job is telling the already-running ollama service to pull a model over the network. The model lands in ollama's own volume, not the puller's, so it's cached across every future docker compose up — confirmed by watching a re-run finish in under a second instead of re-downloading. app waits for that pull to fully complete, not just for Ollama to answer a healthcheck, so there's no race against a half-downloaded model on first boot.
What I'd want to remember from this
- LangChain4j's
AiServices+ tool-calling genuinely removes boilerplate — the agent interfaces stayed clean even once real tool use and structured output were involved. - LangGraph4j's interrupt/resume + Postgres checkpointing is the feature I'd reach for again immediately; building durable human-in-the-loop pauses by hand would've been a project on its own.
- A synchronous call with no timeout is a promise you can't keep once a caller stops waiting — and if nothing can cancel the underlying work, "the caller gave up" and "the work is still running" quietly become two different facts.
- Concurrency is a property of your code's intent; whether it's real depends entirely on what's on the other end of the call.
- For strict-output tasks, a reasoning model's internal monologue is a cost you pay on every call, not a quality feature.
- A few-shot example that's copyable as a complete, plausible answer is a hallucination surface, especially for a model that's unsure what else to do.
- Prompt engineering has a ceiling with small models — it's worth knowing exactly where the framework's own capabilities stop, rather than assuming a wording tweak will eventually close the gap.
The repo — graph, agents, the timeout/fallback fixes, and the Compose setup — is at github.com/ykpraveen/langchain4j-sample.
Top comments (1)
I would probably try the same example in python, as langchain and langgraph are probably more mature. And probably use better models on a gpu pc 😀.