DEV Community: nicholas

Redis Memory Full and Keys Evicted: A Java Backend Checklist of 4 Spots

nicholas — Tue, 03 Nov 2026 15:18:17 +0000

Wednesday afternoon, cache hit rate dropped from 98% to 62%, and database QPS quintupled. Ops checked around and said: Redis memory is maxed out, some keys were evicted, it's normal behavior. But the keys that got evicted happened to be login sessions, and users were collectively kicked offline. Memory full means eviction—sounds perfectly reasonable. The problem is which keys get evicted and why those. This article walks through Redis memory eviction from policy configuration down to kernel behavior, and throws in a set of monitoring code you can run directly.

I. What Exactly Is This Problem

Conclusion: Redis deleting keys after memory is maxed out is not a bug—it's the maxmemory-policy setting doing its job by the rules. Pick the wrong rule, and it won't delete what you wanted deleted.

Redis is an in-memory database—all data lives in memory. Give it a limit, and it won't eat memory indefinitely. That limit is called maxmemory. Whether writes can still go through once you hit the limit, and who gets deleted when they can't, all depends on maxmemory-policy.

Many teams, to "keep things simple" at launch, just copy a chunk of config:

maxmemory 8gb
maxmemory-policy allkeys-lru

allkeys-lru means: when memory runs short, pick the least recently accessed key from all keys and delete it. Note the word "all." Sessions, product cache, rate-limit counters, distributed locks, feature flags—all treated the same. As long as it's the least recently accessed, it gets killed first.

Config-type data suffers the most. Take a feature flag stored in Redis: once written, nobody reads it again. An hour later memory gets tight, and it's the first to go. Then the business logic reads a null value, falls through to the default branch, and the feature quietly turns off—with no error whatsoever.

Redis supports 8 eviction policies in total. The difference comes down to two things: which pool to pick from and by what rule to pick.

PolicyCandidate ScopeSelection RulenoevictionNo evictionWrites return an errorallkeys-lruAll keysLeast recently accessedallkeys-lfuAll keysLowest access frequencyallkeys-randomAll keysRandomvolatile-lruKeys with TTL setLeast recently accessedvolatile-lfuKeys with TTL setLowest access frequencyvolatile-randomKeys with TTL setRandomvolatile-ttlKeys with TTL setShortest remaining TTL

The difference between allkeys-* and volatile-* is fatal. The former deletes your permanent config right along with everything else; the latter only touches data you explicitly marked as "expirable."

There's one more easily overlooked point: the default policy is noeviction. Instances that never configured maxmemory-policy won't delete anything when memory fills up—all write commands return OOM command not allowed when used memory > 'maxmemory'. Reads work fine; writes all fail. Some people treat this as "Redis is stuck," when in reality it's explicitly rejecting you.

To determine whether an instance has ever evicted anything, one metric is enough:

redis-cli info stats | grep evicted_keys

This number is the cumulative count of keys evicted since process start—it only goes up. It's a cumulative value, not a current value. It resets to zero on restart, so don't use it for business analytics. What you actually need to watch is its rate of growth—tens of thousands in half an hour means memory is under sustained pressure, not an occasional blip.

II. What's Really Happening Under the Hood

Conclusion: Redis's memory limit is "effective memory," not "physical memory"; eviction is a check performed before each command execution; LRU is sampled estimation, not exact ordering.

Where Memory Actually Goes

There are a few key fields in INFO memory, and many people only look at the first one:

used_memory: data memory as counted by Redis itself, including the data and the overhead of structures managing that data
used_memory_rss: actual process usage as seen by the OS, including memory fragmentation
used_memory_dataset: pure data portion, minus management overhead
mem_fragmentation_ratio: used_memory_rss / used_memory

A concrete example. You store 1 million string keys, each key averaging 20 bytes and each value averaging 80 bytes—the data itself is roughly 100MB. But Redis attaches a dictEntry and a redisObject to every key, plus jemalloc's fixed-size-bin allocation leaves holes. Actual used_memory often lands around 150MB—50% inflation is normal.

So for the question "My data is only 100MB, why is 8G of memory full?"—the answer usually lies in four areas: management overhead, big keys, replication buffers, and client output buffers—not the data itself.

Also watch the fragmentation ratio:

mem_fragmentation_ratio between 1.0 and 1.5: normal
Greater than 1.8: severe fragmentation, consider enabling activedefrag
Less than 1: some memory has been swapped out, and performance will fall off a cliff

Eviction Happens Before Command Execution

Redis processes commands single-threaded (referring to the main command execution thread). For every command that might increase memory—SET, LPUSH, HSET—before execution it checks once: has current used_memory exceeded maxmemory?

Once it exceeds the limit, it enters the eviction process, deleting keys in a loop until it drops below the limit or can't delete any more. This loop has a cap; it won't delete indefinitely, avoiding a single command getting stuck for too long.

The key point is here: this check is triggered per command, not by a background scheduled job. That is, if there are no writes for a while, memory can exceed the limit without being cleaned up. Conversely, a large batch-write script may repeatedly trigger eviction during execution.

LRU is sampling, not sorting

Textbook LRU maintains a full access linked list, which is costly in both memory and CPU. Redis doesn't do that.

It records a 24-bit timestamp for each key (the lru field, reusing space in redisObject). When evicting, it randomly samples maxmemory-samples keys, 5 by default, and deletes the least recently accessed among those 5.

Sampling 5 is obviously not accurate. But Redis compensates in two ways:

Maintain a 16-element candidate pool (pool), compare each sample against the pool, and keep the "older" ones
Eviction is continuous; if one pass isn't accurate, a few more rounds will make it accurate

Set maxmemory-samples to 10 and accuracy approaches true LRU, at the cost of a bit more CPU per eviction. In production, choose between 5 and 10; don't set it too high.

LFU is a different mechanism. It gives each key an 8-bit counter; each access increments it probabilistically, while decaying keys that haven't been accessed for a long time (lfu-decay-time, default 1 minute). The 8-bit max is 255, and it uses logarithmic growth, so the first few accesses increase quickly and later ones increase slowly.

LFU suits scenarios where "hot spots are stable," such as homepage recommendation slots. LRU suits scenarios where "hot spots drift," such as breaking news. Choosing the wrong one won't crash anything, but hit rate will be noticeably worse.

The nastiest case with volatile policies

Suppose you set the policy to volatile-lru, and in your business code a batch of keys is missing an expiration time.

When memory is full, Redis looks for "keys with a TTL" to evict. If it finds them, it evicts normally. What if it can't find any?

The answer is: it treats that as nothing to evict, and write commands directly return an OOM error.

This is exactly the root cause of many cases where "volatile-lru is set but writes still fail." The policy isn't wrong; the data doesn't meet the policy's precondition. The volatile-* family has a hard precondition: every key that should be eligible for eviction must have a TTL.

Deleting a big key can block the main thread

Evicting a key isn't just removing it from the dictionary; it also has to free the memory it occupies. If the key is a Hash with 5 million elements, freeing it can take tens of milliseconds.

Redis 4.0 and later provides asynchronous freeing:

lazyfree-lazy-eviction yes

Once enabled, eviction only performs a logical delete, and the actual memory freeing is handed off to a background thread. The cost is that memory reclamation is delayed, and used_memory will briefly be higher than the actual value.

Large keys should be split up in the first place. A single value over 10KB is a warning sign, and over 1MB is basically a design problem. Use MEMORY USAGE key to see the actual usage of a single key, and redis-cli --bigkeys to scan the entire database.

Eviction under primary-replica architecture

Only the primary node makes eviction decisions. After the primary deletes a key, it generates a DEL command and syncs it to the replicas.

Replicas do not evict on their own. So if a replica's maxmemory is set the same as the primary's, the replica's memory may exceed the limit. This is also why replicas are usually recommended to be configured with maxmemory-policy noeviction, so they are only responsible for copying the primary's delete actions and do not make decisions themselves.

What should maxmemory be set to

Do not set it to 100% of physical memory.

There are several areas that must be left room for:

Replication buffer (repl-backlog, 1MB by default, but it can spike during a full sync)
AOF rewrite buffer
Client output buffer (Redis 7 and later has maxmemory-clients to limit it separately)

Rule of thumb: set maxmemory to 60%–75% of physical memory. If the machine runs only Redis, 75%; if it is co-located with other services, 60% or even lower. The remaining space will also be consumed in no small part by the child process forked during BGSAVE doing copy-on-write.

3. Hands-on: build an observable cache step by step

Conclusion: data such as sessions must go through volatile-lru + enforced TTL, and there must be code-level eviction monitoring, so you do not have to wait until users are logged out to discover the problem.

The project below is called redis-memory-guard and contains three things: a session cache with TTL, a monitoring component that collects Redis memory metrics every 30 seconds, and an endpoint that exposes the metrics.

Project structure

redis-memory-guard
├── docker-compose.yml
├── pom.xml
└── src/main
    ├── java/com/example/redisguard
    │   ├── RedisMemoryGuardApplication.java
    │   ├── monitor
    │   │   ├── RedisMemoryMonitor.java
    │   │   ├── RedisMemorySnapshot.java
    │   │   └── RedisMonitorController.java
    │   └── session
    │       ├── SessionCacheService.java
    │       └── SessionController.java
    └── resources/application.yml

Start a deliberately misconfigured Redis

First, start an instance with the kind of wrong configuration you are preparing to troubleshoot: a 128MB limit, allkeys-lru, to make it easy to reproduce the problem.

services:
  redis:
    image: redis:8
    container_name: redis-eviction-demo
    command:
      - redis-server
      - --maxmemory
      - 128mb
      - --maxmemory-policy
      - allkeys-lru
    ports:
      - "6379:6379"

Start it:

docker compose up -d

pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>4.1.2</version>
        <relativePath/>
    </parent>

    <groupId>com.example</groupId>
    <artifactId>redis-memory-guard</artifactId>
    <version>1.0.0</version>
    <name>redis-memory-guard</name>
    <description>Redis 内存淘汰监控示例</description>

    <properties>
        <java.version>21</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>4.1.2</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-redis</artifactId>
            <version>4.1.2</version>
        </dependency>

        <!-- Lettuce 连接池需要，不引这个 pool 配置不生效 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-pool2</artifactId>
            <version>2.13.1</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <version>4.1.2</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <version>4.1.2</version>
            </plugin>
        </plugins>
    </build>
</project>

Note the commons-pool2 line. Many people write connection pool parameters in application.yml and find they don't take effect—this dependency is missing. Lettuce uses its own connection pool implementation, and Spring Boot only enables pooling configuration when commons-pool2 is present on the classpath.

application.yml

server:
  port: 8080

spring:
  application:
    name: redis-memory-guard
  data:
    redis:
      host: 127.0.0.1
      port: 6379
      database: 0
      timeout: 2s
      lettuce:
        pool:
          enabled: true
          max-active: 16
          max-idle: 8
          min-idle: 2
          max-wait: 1s

timeout: 2s is the command timeout, not the connection timeout. The default is no timeout, and a slow command can hang a request thread indefinitely, so always set it explicitly in production.

Main Class

package com.example.redisguard;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.scheduling.annotation.EnableScheduling;

@SpringBootApplication
@EnableScheduling
public class RedisMemoryGuardApplication {

    public static void main(String[] args) {
        SpringApplication.run(RedisMemoryGuardApplication.class, args);
    }
}

@EnableScheduling is used by the monitoring component's scheduled task. If you miss it, the @Scheduled method will never execute, not a single log line will appear, and you'll think the monitoring code is wrong.

Session Cache Service

package com.example.redisguard.session;

import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Service;

import java.time.Duration;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;

@Service
public class SessionCacheService {

    private static final String KEY_PREFIX = "session:";
    private static final Duration TTL = Duration.ofMinutes(30);

    private final StringRedisTemplate redisTemplate;

    public SessionCacheService(StringRedisTemplate redisTemplate) {
        this.redisTemplate = redisTemplate;
    }

    public void save(String sessionId, String userId, String nickname) {
        String key = KEY_PREFIX + sessionId;

        Map<String, String> data = new LinkedHashMap<>();
        data.put("userId", userId);
        data.put("nickname", nickname);
        data.put("loginAt", String.valueOf(System.currentTimeMillis()));

        redisTemplate.opsForHash().putAll(key, data);
        // putAll 不会带过期时间，TTL 必须单独设置
        redisTemplate.expire(key, TTL);
    }

    public Map<String, String> find(String sessionId) {
        String key = KEY_PREFIX + sessionId;

        Map<Object, Object> raw = redisTemplate.opsForHash().entries(key);
        if (raw.isEmpty()) {
            return Map.of();
        }

        Map<String, String> result = new LinkedHashMap<>();
        raw.forEach((field, value) -> result.put(String.valueOf(field), String.valueOf(value)));

        // 每次读都续期，做成滑动过期
        redisTemplate.expire(key, TTL);
        return result;
    }

    public void remove(String sessionId) {
        redisTemplate.delete(KEY_PREFIX + sessionId);
    }

    public long ttlSeconds(String sessionId) {
        Long ttl = redisTemplate.getExpire(KEY_PREFIX + sessionId, TimeUnit.SECONDS);
        return ttl == null ? -2L : ttl;
    }
}

Two details are worth mentioning.

After putAll, calling expire separately is required. Redis's HSET command itself does not include an expiration time, and Spring Data Redis does not fold TTL into the putAll wrapper. If you miss this line, the key becomes permanent, and permanent keys are never evicted under volatile-lru—the cache only grows and never shrinks.

The renewal in find is sliding expiration: each access resets the 30-minute window. It suits sessions where the user stays active. If what you want is "must re-authenticate 30 minutes after login," then don't renew; use absolute expiration.

Session Endpoint

package com.example.redisguard.session;

import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.util.Map;

@RestController
@RequestMapping("/session")
public class SessionController {

    private final SessionCacheService sessionCacheService;

    public SessionController(SessionCacheService sessionCacheService) {
        this.sessionCacheService = sessionCacheService;
    }

    @PostMapping("/{sessionId}")
    public Map<String, Object> create(@PathVariable String sessionId,
                                      @RequestParam String userId,
                                      @RequestParam String nickname) {
        sessionCacheService.save(sessionId, userId, nickname);
        return Map.of(
                "sessionId", sessionId,
                "ttlSeconds", sessionCacheService.ttlSeconds(sessionId)
        );
    }

    @GetMapping("/{sessionId}")
    public ResponseEntity<Map<String, String>> get(@PathVariable String sessionId) {
        Map<String, String> session = sessionCacheService.find(sessionId);
        if (session.isEmpty()) {
            return ResponseEntity.notFound().build();
        }
        return ResponseEntity.ok(session);
    }

    @DeleteMapping("/{sessionId}")
    public Map<String, Object> remove(@PathVariable String sessionId) {
        sessionCacheService.remove(sessionId);
        return Map.of("deleted", true);
    }
}

Memory Snapshot

package com.example.redisguard.monitor;

public record RedisMemorySnapshot(long usedMemory,
                                  long usedMemoryRss,
                                  long maxMemory,
                                  long evictedKeys,
                                  long evictedDelta,
                                  String policy) {

    public static RedisMemorySnapshot empty() {
        return new RedisMemorySnapshot(0L, 0L, 0L, 0L, 0L, "unknown");
    }

    public double usedRatio() {
        if (maxMemory <= 0L) {
            return 0d;
        }
        return (double) usedMemory / maxMemory;
    }

    public double fragmentationRatio() {
        if (usedMemory <= 0L) {
            return 0d;
        }
        return (double) usedMemoryRss / usedMemory;
    }

    public String usedMemoryHuman() {
        return human(usedMemory);
    }

    public String maxMemoryHuman() {
        return maxMemory <= 0L ? "未设置" : human(maxMemory);
    }

    private static String human(long bytes) {
        if (bytes < 1024L) {
            return bytes + "B";
        }
        double kb = bytes / 1024.0;
        if (kb < 1024.0) {
            return String.format("%.1fKB", kb);
        }
        double mb = kb / 1024.0;
        if (mb < 1024.0) {
            return String.format("%.1fMB", mb);
        }
        return String.format("%.2fGB", mb / 1024.0);
    }
}

This is a record with immutable fields, suitable for passing between the monitoring thread and HTTP threads. evictedDelta is the "number of evictions added this round," which is much more useful than a cumulative value—a cumulative value only proves that "something was deleted," while a delta tells you that "deletions are happening now."

Monitoring Collection

package com.example.redisguard.monitor;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.core.RedisCallback;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;

import java.util.Properties;

@Component
public class RedisMemoryMonitor {

    private static final Logger log = LoggerFactory.getLogger(RedisMemoryMonitor.class);

    private final StringRedisTemplate redisTemplate;

    private volatile RedisMemorySnapshot latest = RedisMemorySnapshot.empty();
    private volatile long lastEvictedKeys = -1L;

    public RedisMemoryMonitor(StringRedisTemplate redisTemplate) {
        this.redisTemplate = redisTemplate;
    }

    @Scheduled(fixedDelay = 30_000L)
    public void collect() {
        RedisMemorySnapshot snapshot = redisTemplate.execute(
                (RedisCallback<RedisMemorySnapshot>) this::read);

        if (snapshot == null) {
            log.warn("读取 Redis INFO 失败，本次采集跳过");
            return;
        }

        latest = snapshot;

        if (snapshot.evictedDelta() > 0L) {
            log.warn("Redis 正在淘汰 key：本轮新增 {} 个，策略 {}，已用 {} / 上限 {}，碎片率 {}",
                    snapshot.evictedDelta(),
                    snapshot.policy(),
                    snapshot.usedMemoryHuman(),
                    snapshot.maxMemoryHuman(),
                    String.format("%.2f", snapshot.fragmentationRatio()));
        }
    }

    private RedisMemorySnapshot read(RedisConnection connection) {
        Properties memory = connection.serverCommands().info("memory");
        Properties stats = connection.serverCommands().info("stats");
        if (memory == null || stats == null) {
            return null;
        }

        long used = parseLong(memory.getProperty("used_memory"));
        long rss = parseLong(memory.getProperty("used_memory_rss"));
        long max = parseLong(memory.getProperty("maxmemory"));
        long evicted = parseLong(stats.getProperty("evicted_keys"));
        String policy = memory.getProperty("maxmemory_policy", "unknown");

        long delta;
        if (lastEvictedKeys < 0L) {
            delta = 0L;
        } else if (evicted < lastEvictedKeys) {
            // Redis 重启过，计数归零，这一轮整体当作新增
            delta = evicted;
        } else {
            delta = evicted - lastEvictedKeys;
        }
        lastEvictedKeys = evicted;

        return new RedisMemorySnapshot(used, rss, max, evicted, delta, policy);
    }

    private long parseLong(String value) {
        if (value == null || value.isBlank()) {
            return 0L;
        }
        try {
            return Long.parseLong(value.trim());
        } catch (NumberFormatException e) {
            return 0L;
        }
    }

    public RedisMemorySnapshot latest() {
        return latest;
    }
}

What this code does is simple: every 30 seconds it sends two INFO commands over the Redis protocol and parses the text results into a struct. Using RedisCallback instead of creating a new Jedis connection is to reuse Spring's connection pool and timeout configuration, and to route issues through unified exception handling.

serverCommands().info("memory") returns Properties, and the key names are exactly the field names in the INFO output, all lowercase with underscores. evicted_keys is only in INFO stats, so don't look for it in the memory section.

Metrics Endpoint

package com.example.redisguard.monitor;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.Map;

@RestController
@RequestMapping("/admin/redis")
public class RedisMonitorController {

    private final RedisMemoryMonitor redisMemoryMonitor;

    public RedisMonitorController(RedisMemoryMonitor redisMemoryMonitor) {
        this.redisMemoryMonitor = redisMemoryMonitor;
    }

    @GetMapping("/memory")
    public Map<String, Object> memory() {
        RedisMemorySnapshot snapshot = redisMemoryMonitor.latest();
        return Map.of(
                "usedMemory", snapshot.usedMemoryHuman(),
                "maxMemory", snapshot.maxMemoryHuman(),
                "usedRatio", String.format("%.2f%%", snapshot.usedRatio() * 100),
                "fragmentationRatio", String.format("%.2f", snapshot.fragmentationRatio()),
                "policy", snapshot.policy(),
                "evictedKeysTotal", snapshot.evictedKeys(),
                "evictedKeysLastRound", snapshot.evictedDelta()
        );
    }
}

Run and Verify

mvn spring-boot:run

Write a session:

curl -X POST "http://localhost:8080/session/s-1001?userId=1001&nickname=lvdongbin"

Returns:

{"sessionId":"s-1001","ttlSeconds":1799}

Read it back:

curl "http://localhost:8080/session/s-1001"

{"userId":"1001","nickname":"lvdongbin","loginAt":"1792934400000"}

Check monitoring:

curl "http://localhost:8080/admin/redis/memory"

{"usedMemory":"1.2MB","maxMemory":"128.0MB","usedRatio":"0.94%","fragmentationRatio":"1.08","policy":"allkeys-lru","evictedKeysTotal":0,"evictedKeysLastRound":0}

To trigger eviction, just flood Redis with data:

docker exec redis-eviction-demo redis-benchmark -n 200000 -r 200000 -d 2048 -t set -q

This command uses Redis's built-in benchmarking tool to write 200,000 random keys, each value around 2KB, which can easily fill up 128MB. Once you push past the limit, check /admin/redis/memory again; evictedKeysLastRound will no longer be 0, and warnings about Redis 正在淘汰 key will also appear in the logs.

The value of this code is not in the monitoring itself, but in turning "eviction is happening" into a number you can receive a signal from. A drop in cache hit rate is lagging; the eviction count is real-time.

4. Pitfalls and Best Practices

Conclusion: 80% of Redis memory problems come from two things: "keys without TTL" and "maxmemory set too high."

Pitfall 1: HSET, LPUSH and then forgetting EXPIRE

Redis expiration is attached to the key, not to fields. If you keep adding fields to a Hash, the key's own TTL will not be reset by writes, nor will it be created automatically.

A real case: a project's user profile cache used opsForHash().putAll(key, map) in the code and never set an expiration. Three months after launch, this Hash had grown to more than 20 million keys and filled the entire instance. When the memory alarm went off, the first things killed by allkeys-lru were login sessions.

The fix is not to add memory, but to add one line, expire, where the write happens. For every method that writes to cache, TTL must be a required parameter and must not have a default value.

Pitfall 2: maxmemory set to 100% of physical memory

What happens when it fills up? used_memory indeed won't exceed the limit, but used_memory_rss will. The child process forked during BGSAVE performs copy-on-write and needs extra physical memory. Linux's OOM Killer will kill the Redis process, leaving a Killed in the logs, with no Redis error message of its own.

More insidious is swap. When physical memory is insufficient, the system swaps Redis pages to disk, and mem_fragmentation_ratio will drop below 1. At this point Redis can still run, but every access to cold data is disk I/O, and latency goes from 0.2 ms to 20 ms. Monitoring shows no errors, and it just feels like "Redis has gotten slower."

Pitfall 3: Looking only at used_memory, not at used_memory_rss

Redis freed 1GB of data, and used_memory drops immediately, but used_memory_rss does not necessarily drop. jemalloc does not always return memory to the operating system.

This does not count as a leak. As long as the fragmentation rate stays within 1.5, there is no need to worry about it. But if used_memory_rss keeps rising while used_memory does not increase, then it is real fragmentation. Enable activedefrag yes, or find an off-peak period and restart once.

Pitfall 4:volatile-lru configured with a bunch of keys without TTL

We covered the mechanism earlier; now let's look at the symptom. It goes like this: memory isn't full, yet write commands return OOM command not allowed. Ops takes one look at maxmemory, sees there's still headroom, and is completely baffled.

One command to troubleshoot:

redis-cli info keyspace

The output has two key columns per db: the total count keys and the number of keys with a TTL expires. If expires is 0 or far smaller than keys, then the volatile-* policy is basically equivalent to noeviction.

Best practices checklist:

Always use allkeys-lru for cache data, and give every key a TTL — belt and suspenders
When co-locating cache with "data that must not be lost," use volatile-lru, and also run a script periodically to scan for keys without a TTL
Put sessions, verification codes, and rate-limit counters in a separate instance or a separate db — don't let them compete with business cache for memory
With maxmemory, leave at least 25% of physical memory for fork, replication buffers, and client buffers
Enable lazyfree-lazy-eviction yes to avoid blocking the main thread when deleting large keys
Wire the growth rate of evicted_keys into alerting, with thresholds set per workload — for example, alert if more than 1000 are added within 5 minutes

The most critical item on the list is this one. The vast majority of Redis memory incidents are not due to a lack of monitoring, but because monitoring only looks at memory usage and not eviction rate.

V. Performance Comparison and Technology Selection

Conclusion: The CPU overhead differences between eviction policies are very small. The key to selection is "whether the data can be lost," not "which one is faster."

Comparison of eviction overhead across different policies:

PolicyExtra overhead per evictionHit rate performanceUse casenoevictionNone (directly rejects writes)Not applicableStoring data that cannot be lost, such as queues and configurationsallkeys-lruSamples 5 keys and compares timestampsGood for hotspot drift scenariosPure cache instancesallkeys-lfuSampling + counter comparisonBetter for stable hotspot scenariosRecommendation slots, leaderboardsallkeys-randomAlmost zeroWorst, not recommendedAlmost never usedvolatile-lruSame as allkeys-lruDepends on TTL coverageMixed deployment of cache and persistent datavolatile-ttlCompares remaining TTLModerateScenarios with clear time-based eviction

A few empirically measured rules of thumb:

maxmemory-samples Adjusting from 5 to 10 improves the hit rate by about 1~3 percentage points, while CPU overhead increases by less than 1%. It is worth tuning.

lazyfree-lazy-eviction Once enabled, the latency of evicting large keys drops from tens of milliseconds to the microsecond level, at the cost of delayed memory reclamation by a few seconds. It is recommended to enable it in high-concurrency write scenarios.

activedefrag Continuously consumes CPU (by default occupying one background thread). Do not enable it when the fragmentation rate is below 1.5.

For the decision order in technology selection, I suggest the following:

Would losing this data cause a business failure? If yes → noeviction, use a separate instance.
If not, but it is mixed with other data in the same instance? → volatile-lru, force all cache keys to have a TTL.
Is the entire instance used as cache? → allkeys-lru, simple and direct.

Do not choose LFU just to be "more precise." The counter decay parameter in LFU is very difficult to tune. If it is not tuned well, "new hotspots can never get in" because the counters of old keys are too high. Unless you have a clear judgment that hotspots are stable, LRU is more reliable.

VI. Summary

Deleting keys after Redis memory is full is designed behavior, not a failure. The real questions to ask are three: which batch of keys is being deleted, why them, and will critical data be deleted next time.

maxmemory-policy determines the candidate pool. allkeys-* deletes permanent data along with everything else, while volatile-* only touches keys that have a TTL. Choosing volatile-* without setting a TTL on your keys turns the instance into noeviction, and every write request returns an error.

Don't push maxmemory all the way to your physical memory limit. Leave at least 25% for fork, replication buffers, and client buffers; otherwise the OOM killer will make a much harsher decision for you at some ungodly hour.

For monitoring, used_memory only tells you "almost full," while the growth rate of evicted_keys tells you "eviction has already started." Wiring the latter into your alerts is far more useful than staring at memory usage.

At the code level, a TTL should be a required field when writing to the cache, not an optional one. Only if you hold this line does volatile-lru mean anything.

References

LLM JSON Parsing Always Fails? Java Structured Output in Practice

nicholas — Tue, 03 Nov 2026 15:15:43 +0000

A week after the customer service ticket extraction feature launched, the logs were full of JsonParseException. Sometimes the model wraps JSON in a Markdown code block; sometimes it prepends a line like "Sure, here are the results"; and the field name can even change from orderNo to order_no. Adding regexes, truncating strings, and tweaking prompts holds up for a couple of days before it breaks again. This article explains LangChain4j's structured output: how to push format constraints down to the protocol layer, and how to add business validation that the schema cannot handle. The code is a complete project; copy it and it runs.

1. What This Problem Actually Is: Why Does JSON Returned by LLMs Always Break

JSON parsing failures are not because the model is "disobedient"; they happen because you only ask it with prompts and do not constrain it at the protocol layer.

The five most common ways of asking a model to return JSON go wrong:

Wrapped in a code block: it returns a json block wrapped in three backticks, and Jackson immediately reports "Unexpected character '`'".
Prefixes and suffixes: "Sure, I've organized it for you as follows:" followed by JSON. You slice from { to }, and it breaks again when curly braces appear inside string values.
Field drift: this time orderNo, next time order_no, and the time after that order_number.
Type drift: the amount should be a number, but the model gives you "399 元", and parsing BigDecimal blows up immediately.
Enums improvising: you define REFUND and LOGISTICS, and it returns "退款" and "refund_request".

Many people's first reaction is to change the prompt: add "must return pure JSON", add "do not explain", add few-shot examples. It works in the short term, then breaks again a couple of days later. The reason is no mystery:

Prompts are soft constraints. When generating each token, the model can still choose any word.
As context grows longer, earlier format requirements get diluted. Ticket conversations are often thousands of words long, so the model is more likely to "casually" add an explanation.
Different models understand "JSON" differently. Some think code blocks count as JSON; some write numbers as strings.
Even low temperature is not zero probability. As long as one token goes off track, the whole response is ruined.

Some people also think that setting temperature to 0 makes things stable. Temperature only affects the sampling distribution; it does not guarantee that the same input always produces the same output: server-side batching, model version canary releases, and changes in context length can all make results drift. What is more troublesome is "half-correct" output—four out of five fields are correct, parsing succeeds, and bad data is silently written to the database. A total failure at least triggers an alert; a half-correct result quietly pollutes reports.

Hand-written cleanup code is equally unreliable. Regexes can only cover the failure modes you have seen; substring(indexOf("{"), lastIndexOf("}")) truncates the JSON when } appears inside a string value; worse, swallowing parse exceptions and returning null pushes the null pointer to the business layer and doubles the debugging cost.

Cleanup code also has a hidden cost: it mixes "format problems" and "business problems" together. When parsing fails, you add a regex; when a field is wrong, you add if; six months later nobody dares to touch that code, because nobody knows which line is compensating for which model's which quirk. Structured output pulls this back into the type system: if something breaks, it is a compile error or a parse exception, and the boundary is clear.

Here the problem needs to be split into two layers:

LayerSpecific problemWho can solve itShapeIs it valid JSON, are the field names correct, are the types correct, are the enum values validJSON SchemaSemanticsIs the amount a refund amount or an order amount, is the classification correct, is the order number the one for this orderBusiness validation

The schema can guarantee that amount is a number, but it cannot guarantee that the number comes from "refund 399 yuan" rather than "original price 599 yuan". The schema can guarantee that category is one of the four enums, but it cannot guarantee that the model did not classify a logistics issue as a refund.

In one sentence: leave format to the schema, facts to validation, and failures to bounded retries.

2. How the Underlying Mechanism Actually Works: How JSON Schema Constrains LLM Output

Structured output compiles Java types into a JSON Schema, sends it to the model with the request, constrains the generation phase with the schema, and then deserializes the returned text back into an object.

The whole chain has three steps.

Step 1: Translate Java Classes into a Schema

LangChain4j uses reflection to read your return type and generates the schema field by field:

String → "type": "string"
int / Integer → "type": "integer"
BigDecimal / double → "type": "number"
enum → "type": "string" plus the enum whitelist
List<T> → "type": "array" plus items
Nested objects → "type": "object"; recursive structures are closed off with $defs and $ref

The @Description on a field gets written into the schema's description. Don't underestimate this one line of text — it's mainly what the model relies on to decide "what should go in this field." The @JsonProperty(required = true) on a field goes into the required list.

This translation is recursive. List<TicketItem> produces an array, and the element objects get expanded in turn; self-referencing types don't recurse infinitely — once LangChain4j detects a cycle, it switches to $ref pointing at $defs. So you can put tree structures in your DTO and still get a valid schema, it's just that the deeper the nesting, the higher the odds the model fills something in wrong.

A schema generated from a ticket class looks roughly like this:

{
  "type": "object",
  "properties": {
    "orderNo": {
      "type": "string",
      "description": "订单号，格式为 ORD- 加 8 位数字"
    },
    "category": {
      "type": "string",
      "enum": ["REFUND", "LOGISTICS", "INVOICE", "OTHER"]
    },
    "amount": {
      "type": "number",
      "description": "涉及金额，单位元；对话里没提金额就填 0"
    }
  },
  "required": []
}

Note that required here is empty. Only fields marked @JsonProperty(required = true) make it in. Before sending to OpenAI, strict mode does another rewrite pass: it stuffs all the keys into required and turns the types of fields not marked required into nullable forms, such as ["string", "null"] and ["number", "null"]. So "required" under strict mode only guarantees the key is present, not that the value is non-null. I'll dig into this trap later.

Step 2: Put the Schema into the Request

The request carries the schema via ResponseFormat:

ResponseFormat.builder()
        .type(ResponseFormatType.JSON)
        .jsonSchema(schema)
        .build();

The OpenAI adapter layer translates it into response_format: {"type": "json_schema", ...}.

If strict is enabled, the server enforces the constraint at the decoding stage: for every token generated, candidates that violate the schema get masked out outright. The model isn't "trying its best to get it right" — it "can't write it wrong." That's a whole different thing from asking it nicely in a prompt.

Capabilities vary widely across vendors:

OpenAI's Structured Outputs supports a common subset of JSON Schema, with hard constraints in strict mode.
Inference frameworks like vLLM use guided decoding and can also mask tokens according to the schema.
Plenty of OpenAI-compatible services only support json_object. It only guarantees "the whole thing is valid JSON" — field names, types, and enums are all ignored.
Some services just ignore response_format entirely — whatever you pass, they act like they never saw it.

json_object and json_schema often get conflated. The former only guarantees syntax; the latter governs field names and types as well. A concrete example: you ask it to extract {"amount": 399}, and json_object mode might return {"退款金额": "399元"} — perfectly valid syntax, and your DTO still can't handle it. Schema mode won't even let the key name amount be changed.

LangChain4j uses capability flags to decide which path to take. Only if the model's supportedCapabilities() includes RESPONSE_FORMAT_JSON_SCHEMA does it go with native schema; otherwise it appends the format instructions to the user message. The fallback mode isn't completely useless — it pastes the schema and field descriptions into the user message, so the structure the model sees is accurate, and most of the time it gets it right. The difference is between "most of the time" and "always."

There's a well-hidden trap here: OpenAiChatModel doesn't declare this capability by default. You have to explicitly declare .supportedCapabilities(RESPONSE_FORMAT_JSON_SCHEMA) for it to turn the capability flag on. Without that line, AiServices will still convert the result into an object, but under the hood it's "pasting a format requirement into the prompt," and stability drops off immediately. A lot of people say "I'm using AiServices, so why does it still fall over?" — and the answer is usually right here.

Step 3: Turn the Returned Text Back into an Object

After AiServices gets the response, it hands it to PojoOutputParser. The parser doesn't just readValue right away — it first tries the whole string; if that fails, it scans backwards for the last } or ], then matches braces going backwards, skipping over braces that sit inside strings. So code fences and surrounding chatter all get rescued.

This leniency is a safety net, not a design goal. If field names or types are wrong, it can't save you either — it just throws OutputParsingException.

There's another case it can't do anything about: the response gets truncated by max_tokens. The JSON is missing a closing brace, brace matching fails, and it throws an exception outright. You can catch this kind of failure early by checking finishReason — don't wait for Jackson to blow up. Setting max_tokens comfortably above your normal output size is a cheap and effective fix.

Three Things Schema Can't Cover

Semantics. Schema governs shape. Whose amount it is, whether the category is right, whether the order number belongs to this order — it doesn't concern itself with any of that.

Unknown values. The OpenAI adapter's strict mode puts all properties into required and also changes the types of fields not marked required into nullable form. The key must be present, and the value can be null. When the model encounters "the amount isn't mentioned in the conversation," it is more likely to give you null than to make up a number. So in the DTO, use wrapper types to receive it. If you use int, Jackson will silently turn null into 0, and validation sees fake data; if the business does not accept nulls, check it in validation, or add UNKNOWN to the enum and agree on 0 for the amount.

Cross-field rules. A constraint like "the refund amount cannot be greater than the order amount" can be written with conditional expressions in JSON Schema, but support varies widely among model vendors. Don't pin business rules on the schema; just write Java validation.

3. Hands-On: Writing LangChain4j Structured Output Code Step by Step

A minimal working project consists of four parts: dependencies, DTOs, the AI Service interface, and validation plus retry. The three examples below go from simple to complete, and each one is a full class.

Dependencies

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>structured-output-demo</artifactId>
    <version>1.0.0</version>

    <properties>
        <maven.compiler.release>21</maven.compiler.release>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j</artifactId>
            <version>1.22.0</version>
        </dependency>
        <dependency>
            <groupId>dev.langchain4j</groupId>
            <artifactId>langchain4j-open-ai</artifactId>
            <version>1.22.0</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.22.4</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>2.0.20</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.16.0</version>
            </plugin>
        </plugins>
    </build>
</project>

Example 1: AiServices Automatically Generates the Schema

This is the least effort approach. Define an interface, set the return type to a DTO, and LangChain4j handles schema generation, request sending, and result parsing.

package com.example.structured;

import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.openai.OpenAiChatModel;
import dev.langchain4j.model.output.structured.Description;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.UserMessage;

import java.math.BigDecimal;

import static dev.langchain4j.model.chat.Capability.RESPONSE_FORMAT_JSON_SCHEMA;

public class TicketExtractionDemo {

    public enum Category {
        REFUND, LOGISTICS, INVOICE, OTHER
    }

    public static class Ticket {
        @Description("订单号，格式为 ORD- 加 8 位数字")
        public String orderNo;

        @Description("问题分类：REFUND 退款、LOGISTICS 物流、INVOICE 发票、OTHER 其他")
        public Category category;

        @Description("涉及金额，单位元；对话里没提金额就填 0")
        public BigDecimal amount;

        @Description("优先级，1 最低，5 最高；催得急或涉及投诉填 4 以上")
        public Integer priority;

        @Description("用户诉求摘要，不超过 30 个字")
        public String summary;
    }

    public interface TicketAssistant {
        @UserMessage("""
                从下面的客服对话中抽取工单信息。
                只抽对话里明确出现的内容，不要推测。

                对话：
                {{it}}
                """)
        Ticket extract(String dialogue);
    }

    public static void main(String[] args) {
        ChatModel model = OpenAiChatModel.builder()
                .apiKey(System.getenv("OPENAI_API_KEY"))
                .modelName("gpt-4.1-mini")
                // 这两行决定走原生 JSON Schema，而不是在 prompt 里贴格式说明
                .supportedCapabilities(RESPONSE_FORMAT_JSON_SCHEMA)
                .strictJsonSchema(true)
                .temperature(0.0)
                .build();

        System.out.println("模型能力: " + model.supportedCapabilities());

        TicketAssistant assistant = AiServices.builder(TicketAssistant.class)
                .chatModel(model)
                .build();

        Ticket ticket = assistant.extract("""
                用户：我上周买的耳机，订单 ORD-20261012，到现在还没发货。
                客服：我帮您查一下物流。
                用户：不用查了，直接退款，399 元，太慢了。
                """);

        System.out.println("订单号: " + ticket.orderNo);
        System.out.println("分类: " + ticket.category);
        System.out.println("金额: " + ticket.amount);
        System.out.println("优先级: " + ticket.priority);
        System.out.println("摘要: " + ticket.summary);
    }
}

Let's break down the key behaviors:

{{it}} in @UserMessage is a placeholder corresponding to the method's only parameter. You don't need to manually concatenate strings.
The return type is Ticket, not String. This single signature tells LangChain4j to generate a schema and deserialize the result.
@Description is not a comment; it goes into the schema. The more abstract the field name, the clearer the description must be. For example, if priority doesn't say '1 to 5', the model will happily give you 10.
temperature(0.0) reduces randomness, but don't expect it to replace the schema.
System.out.println(model.supportedCapabilities()) is for troubleshooting. If the output lacks RESPONSE_FORMAT_JSON_SCHEMA, you're in prompt fallback mode.

Example 2: Hand-Write the Schema Without Going Through the Interface Proxy

Some scenarios are not suitable for AiServices: the schema is assembled at runtime, you need to reuse a ChatModel, or you want to see exactly what request was sent. In these cases, construct ChatRequest directly.

package com.example.structured;

import dev.langchain4j.data.message.SystemMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.request.ResponseFormat;
import dev.langchain4j.model.chat.request.ResponseFormatType;
import dev.langchain4j.model.chat.request.json.JsonObjectSchema;
import dev.langchain4j.model.chat.request.json.JsonSchema;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.model.openai.OpenAiChatModel;

import java.util.List;

import static dev.langchain4j.model.chat.Capability.RESPONSE_FORMAT_JSON_SCHEMA;

public class RawSchemaDemo {

    public static void main(String[] args) {
        ChatModel model = OpenAiChatModel.builder()
                .apiKey(System.getenv("OPENAI_API_KEY"))
                .modelName("gpt-4.1-mini")
                .supportedCapabilities(RESPONSE_FORMAT_JSON_SCHEMA)
                .strictJsonSchema(true)
                .temperature(0.0)
                .build();

        JsonSchema schema = JsonSchema.builder()
                .name("Ticket")
                .rootElement(JsonObjectSchema.builder()
                        .addStringProperty("orderNo", "订单号，格式为 ORD- 加 8 位数字")
                        .addEnumProperty("category", List.of("REFUND", "LOGISTICS", "INVOICE", "OTHER"))
                        .addNumberProperty("amount", "涉及金额，单位元，没有就填 0")
                        .addIntegerProperty("priority", "优先级，1 到 5")
                        .addStringProperty("summary", "用户诉求摘要，不超过 30 个字")
                        .required("orderNo", "category", "amount", "priority", "summary")
                        .additionalProperties(false)
                        .build())
                .build();

        ChatRequest request = ChatRequest.builder()
                .messages(
                        SystemMessage.from("你是客服工单抽取器，只输出 JSON。"),
                        UserMessage.from("订单 ORD-20261012 一直没发货，我要退款 399 元。"))
                .responseFormat(ResponseFormat.builder()
                        .type(ResponseFormatType.JSON)
                        .jsonSchema(schema)
                        .build())
                .build();

        ChatResponse response = model.chat(request);
        System.out.println(response.aiMessage().text());
    }
}

JsonObjectSchema.builder() provides a series of convenience methods; addStringProperty, addEnumProperty, and addNumberProperty all take two parameters: "name + description." required specifies which fields must be present, and additionalProperties(false) prevents the model from adding extras.

The advantage of hand-writing the schema is control: field names, enum values, descriptions, and required fields are all in your hands. The downside is that it easily drifts out of sync with the DTO. Change one side and forget the other, and parsing will still break. My advice: for scenarios the interface proxy can cover, prefer Example 1; only hand-write when the schema needs to be generated dynamically.

Example 3: Business Validation + Limited Retry

The first two examples solve shape; this one solves semantics. Core idea: after parsing succeeds, don't return immediately—run the rules first. If there's a problem, feed "which rule failed" back to the model and have it rewrite. Retry at most twice.

package com.example.structured;

import com.fasterxml.jackson.databind.ObjectMapper;
import dev.langchain4j.data.message.SystemMessage;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.request.ResponseFormat;
import dev.langchain4j.model.chat.request.ResponseFormatType;
import dev.langchain4j.model.chat.response.ChatResponse;
import dev.langchain4j.model.openai.OpenAiChatModel;
import dev.langchain4j.model.output.structured.Description;
import dev.langchain4j.service.output.JsonSchemas;

import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Pattern;

import static dev.langchain4j.model.chat.Capability.RESPONSE_FORMAT_JSON_SCHEMA;

public class ValidatedExtractor {

    private static final Pattern ORDER_NO = Pattern.compile("^ORD-\\d{8}$");
    private static final ObjectMapper MAPPER = new ObjectMapper();
    private static final int MAX_ATTEMPTS = 3;

    private final ChatModel model;

    public ValidatedExtractor(ChatModel model) {
        this.model = model;
    }

    public static class Ticket {
        @Description("订单号，格式为 ORD- 加 8 位数字")
        public String orderNo;

        @Description("问题分类：REFUND 退款、LOGISTICS 物流、INVOICE 发票、OTHER 其他")
        public String category;

        @Description("退款或涉及金额，单位元；没有就填 0")
        public BigDecimal amount;

        @Description("优先级，1 最低，5 最高")
        public Integer priority;

        @Description("用户诉求摘要，不超过 30 个字")
        public String summary;
    }

    public Ticket extract(String text) {
        ResponseFormat format = ResponseFormat.builder()
                .type(ResponseFormatType.JSON)
                .jsonSchema(JsonSchemas.jsonSchemaFrom(Ticket.class).orElseThrow())
                .build();

        List<String> problems = List.of();

        for (int attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
            String userText = text;
            if (!problems.isEmpty()) {
                userText = text
                        + "\n\n上一次输出有以下问题，请逐条修正后重新输出：\n- "
                        + String.join("\n- ", problems);
            }

            ChatRequest request = ChatRequest.builder()
                    .messages(
                            SystemMessage.from("只做信息抽取。不确定的字段按默认值填，不要编造。"),
                            UserMessage.from(userText))
                    .responseFormat(format)
                    .temperature(0.0)
                    .build();

            ChatResponse response = model.chat(request);
            String raw = response.aiMessage().text();

            try {
                Ticket ticket = MAPPER.readValue(raw, Ticket.class);
                problems = validate(ticket);
                if (problems.isEmpty()) {
                    return ticket;
                }
                System.out.println("第 " + attempt + " 次校验失败: " + problems);
            } catch (Exception e) {
                problems = List.of("输出不是合法 JSON：" + e.getMessage());
                System.out.println("第 " + attempt + " 次解析失败，原始输出：" + raw);
            }
        }

        throw new IllegalStateException(
                "连续 " + MAX_ATTEMPTS + " 次没有拿到合格结果，最后的问题：" + problems);
    }

    private List<String> validate(Ticket t) {
        List<String> problems = new ArrayList<>();

        if (t.orderNo == null || !ORDER_NO.matcher(t.orderNo).matches()) {
            problems.add("orderNo 必须是 ORD- 加 8 位数字，当前是 " + t.orderNo);
        }
        if (t.category == null
                || !List.of("REFUND", "LOGISTICS", "INVOICE", "OTHER").contains(t.category)) {
            problems.add("category 只能是 REFUND、LOGISTICS、INVOICE、OTHER");
        }
        if (t.amount == null
                || t.amount.signum() < 0
                || t.amount.compareTo(new BigDecimal("100000")) > 0) {
            problems.add("amount 必须在 0 到 100000 之间");
        }
        if (t.priority == null || t.priority < 1 || t.priority > 5) {
            problems.add("priority 必须在 1 到 5 之间");
        }
        return problems;
    }

    public static void main(String[] args) throws Exception {
        ChatModel model = OpenAiChatModel.builder()
                .apiKey(System.getenv("OPENAI_API_KEY"))
                .modelName("gpt-4.1-mini")
                .supportedCapabilities(RESPONSE_FORMAT_JSON_SCHEMA)
                .strictJsonSchema(true)
                .temperature(0.0)
                .build();

        ValidatedExtractor extractor = new ValidatedExtractor(model);
        Ticket ticket = extractor.extract(
                "订单 ORD-20261012 一直没发货，我要退款 399 元。");

        System.out.println(MAPPER.writerWithDefaultPrettyPrinter().writeValueAsString(ticket));
    }
}

There are several design points in this code worth explaining.

The schema only declares the shape; validation rules are written in Java. category in Ticket deliberately uses String instead of an enum. In native schema mode, it is an enum whitelist; when third-party compatible services ignore the schema, it is just a free-form string. Validation catches both cases.

Feed the failure reason back. Just saying "regenerate" is useless; the model doesn't know what went wrong. Tell it "orderNo must be ORD- followed by 8 digits, and the current value is 20261012," and the second attempt will basically get it right. This is what separates effective retries from ineffective ones.

Cap the retries. If it still fails after three attempts, the input itself is the problem, and continuing to burn tokens is pointless. Throw an exception directly and hand it off to a human or a dead-letter queue.

The raw output must be logged. When parsing fails, raw is the only clue. In production, wiring this log into alerting is far more useful than guessing afterward why the model went off the rails.

One retry is one full call. Every retry resends the system prompt, the user's original text, and the schema, so the cost is very real. Therefore, list all validation rules at once; don't drip-feed them one by one.

4. Pitfalls and Best Practices: The 8 Places Where Structured Output Most Easily Goes Wrong

Eighty percent of structured output pitfalls aren't in schema syntax; they're in "assuming it works as soon as you configure it."

Just writing "return JSON". That's not a constraint. Either use ResponseFormat + JsonSchema, or accept that it'll add code blocks whenever it feels like it.
Assuming AiServices will always use the native schema. If the model doesn't declare RESPONSE_FORMAT_JSON_SCHEMA, the framework falls back to prompt mode. For OpenAI models, remember to add .supportedCapabilities(RESPONSE_FORMAT_JSON_SCHEMA), and print supportedCapabilities() once before going live to confirm.
You also want optional fields in strict mode. OpenAI’s strict mode puts every property into required, while fields not marked as required allow null. The keys are always there; the values may be empty. On the Java side, use wrapper types (Integer, BigDecimal). If you use int, Jackson will silently convert null to 0, so validation ends up with fake data—harder to track down than an error. If you really don’t want nulls, add UNKNOWN to enums, agree on 0 for amounts, and handle it in validation.
For amounts, use double. Floating-point errors in monetary amounts are a disaster. Use BigDecimal in the DTO and number in the schema, and Jackson can convert them correctly.
Use date fields directly as LocalDate. When generating the schema, LocalDate isn't treated as string; instead, it gets translated into an object with no fields, leaving the model to fill it in blindly. The ObjectMapper on the parsing side also isn't guaranteed to have JavaTimeModule registered. Accept String in the DTO, then convert it to LocalDate in the business layer. Two extra lines save you an afternoon of debugging.
The prompt and the schema each say something different. The prompt pastes the field structure again, so the model may output according to the old version in the prompt. The prompt should only cover the business rules (what counts as a refund, how priority is determined), and leave the structure to the schema.
The DTO is both wide and deep. Twenty fields and three levels of nesting — the schema alone takes up several hundred tokens, and the model also tends to miss fields. Split it into two extractions: first extract the order number and category, then extract the details as needed. Both accuracy and cost improve.
Retries with no cap and no idempotency. Retrying endlessly on validation failure means that with input that never passes, the bill can go up tenfold. Cap it at two or three attempts, log the raw output and issue list for each attempt, and make writes idempotent.

A few more that are less common but potentially fatal:

Field names are too casually chosen. With names like a, b, and tmp, the model can only guess. Use semantic English names such as orderNo and refundAmount, and add @Description to fill in the details.
Forgot additionalProperties(false). The model can add arbitrary fields to the object. These fields cannot enter the DTO, waste output tokens for nothing, and also show that it did not strictly follow your structure. Turn it off when writing the schema by hand.
Streaming output paired with a POJO return value. The streaming method in AiServices returns TokenStream and cannot directly return a DTO. If you want structured results, do not enable streaming, or parse the result yourself after streaming is complete.
Third-party compatible services silently ignore response_format. It does not report an error and still returns a passage of prose. So validation and parsing fallbacks are not optional; they are mandatory.

After launch, monitor three metrics: parse success rate, validation failure rate, and average retry count. If the parse success rate is low, check the model's capability flags and max_tokens first; if the validation failure rate is high, the business rules in the prompt aren't clearly defined; if retry counts rise, it's usually because the model version changed or the input distribution shifted. These three metrics are far more useful than "it feels less accurate lately."

5. Performance Comparison and Technology Selection: JSON Mode, JSON Schema, or Function Calling

For extraction tasks in production, prefer JSON Schema; use Function Calling only when you want the model to decide whether to call a tool.

Approach	What it guarantees	Extra overhead	Suitable scenarios
Plain text + custom parsing	Guarantees nothing	None	One-off scripts, toy demos
JSON mode	The output as a whole is valid JSON	Very small	Older models; compatible services that only support json_object
JSON Schema	Field names, types, enums, required fields	The schema consumes input tokens; decoding is slightly slower	Production extraction; the preferred choice
Function Calling	Tool parameter structure, and the model decides whether to call a tool	Tool definitions consume tokens	Agents, where the model needs to choose actions autonomously

The bulk of the latency is always output tokens and network round trips. The overhead from constrained decoding is barely perceptible on outputs of a few hundred tokens, like ticket extraction. What really sends costs out of control is retries: one failed retry means double the tokens and double the latency. So get the schema right the first time, and list every validation rule up front.

Schemas aren't free. They ride along with every request — the more fields and the deeper the nesting, the higher your input token count. Keeping it to ten fields per DTO and no more than two levels of nesting is a comfortable range. Beyond that, consider splitting the call rather than cramming everything in.

Choosing is simple:

Internal systems doing information extraction, classification, or scoring: JSON Schema + Java validation, no hesitation.
Hooking up to a third-party OpenAI-compatible service: first probe whether it supports response_format; if not, fall back to JSON mode — and not a single line of validation logic can be skipped.
The model needs to decide which tool to call based on user intent: Function Calling. At heart it's also structured output, except what gets structured is "actions."
You just want nicer-looking logs: don't bother — regex extraction of key fields is enough.

6. Summary

Structured output comes down to three sentences:

Shape goes to the schema. Use ResponseFormat with JsonSchema to push the constraints down to the protocol layer; returning a POJO from AiServices auto-generates the schema, but make sure the model actually declares the capability.

Facts go to validation. The schema guarantees amount is a number; it can't guarantee it's a refund amount. Order number format, amount ranges, enum whitelists, cross-field relationships — hard-code all of it in Java.

Failures go to bounded retries. Feed back to the model exactly which rule failed, at most two or three times, and log the raw output. Past that, give up — don't burn tokens out of spite.

The implementation checklist is six items: narrow the DTO, turn on the schema, think through strict mode, always write validation, cap the retries, keep the raw logs complete.

When should you not use structured output? Open-ended copywriting, long-form summarization, brainstorming. These tasks have no stable fields; forcing a schema on them just ties the model's hands and leaves the output flat. Structured output is good for "moving information around," not for "creating content."

One more note for anyone about to ship: build a regression set from twenty real tickets and run it every time you swap models, tweak prompts, or touch the schema. Models aren't compilers — an upgrade won't tell you what's incompatible.

The model writes the answers; the code grades the paper. If the grading rules aren't clear, no model — however large — will save you.