My AI Agent Kept Compressing the Same Conversation. Here's How I Fixed the Anti-Thrashing Bug.

I noticed something weird. Every time my AI agent's process restarted, it would compress the conversation again — even though the last five compressions had been useless.

The prompt was already clean. The system prompt and tool schemas added up to 30K tokens of incompressible floor. Shrinking the message history wasn't going to help. But the agent didn't know that. It ran the compaction loop every single time, burning tokens and time for zero benefit.

The root cause? A single in-memory counter that disappeared on restart.

The Problem

Context compression is how long-running AI agent conversations stay under the model's context window. You have a threshold — say 50K tokens. When the provider reports the prompt approaching that limit, you compact: summarize old messages into a single system-level entry, drop the full history, and keep rolling.

Simple, right?

But there's a corner case: when the system prompt and tool schemas alone are already close to the threshold, compaction can't help. The message history shrinks, sure, but the total prompt stays over the line. So the next turn triggers compaction again. And again. And again.

To stop this, I added a thrashing guard — a counter that ticks up each time compaction runs but doesn't clear the threshold. After two ineffective rounds, it blocks further compression. Clean.

# The anti-thrashing guard (before the fix)
class CompressionState:
    def __init__(self):
        # In-memory only — vanishes on restart
        self._ineffective_compression_count = 0

    def update_from_response(self, usage):
        """Called after each API response with real token counts."""
        if self._verify_compaction_cleared_threshold:
            if self.last_prompt_tokens >= self.threshold_tokens:
                self._ineffective_compression_count += 1
            else:
                self._ineffective_compression_count = 0

    def should_compress(self):
        """Gate check — blocks compression after 2 strikes."""
        return self._ineffective_compression_count < 2

The guard works perfectly in a single session. But it lives in memory. When the process restarts — deployment, crash recovery, even a graceful restart — the counter resets to zero. The guard disarms. And the agent compresses the already-compacted conversation one more time.

The Fix

The pattern was already in the codebase for two other counters: the compression failure cooldown (which prevents retrying a failing provider) and the fallback streak (which tracks how many deterministic fallback summaries were inserted in a row). Both persisted through a durable session-state channel backed by SQLite.

I applied the same pattern to the anti-thrashing counter. Three pieces:

1. A database column. I added compression_ineffective_count to the sessions table, with accessor methods that return the value or write it back.

# hermes_state.py — persistent counter accessors
def get_compression_ineffective_count(self, session_id: str) -> int:
    row = self._conn.execute(
        "SELECT compression_ineffective_count FROM sessions WHERE session_id = ?",
        (session_id,)
    ).fetchone()
    return row[0] if row else 0

def set_compression_ineffective_count(self, session_id: str, count: int) -> None:
    self._conn.execute(
        "UPDATE sessions SET compression_ineffective_count = ? WHERE session_id = ?",
        (count, session_id)
    )

2. A centralized verdict recorder. Every time the update_from_response() method decides whether the last compaction was effective or not, it routes through _record_ineffective_compression_verdict(). This method updates both the in-memory counter and the database row — atomically, in the same code path.

def _record_ineffective_compression_verdict(self, was_ineffective: bool):
    if was_ineffective:
        self._ineffective_compression_count += 1
    else:
        self._ineffective_compression_count = 0

    # Persist through the same channel used by every other durable guard
    setter = getattr(self._session_db, "set_compression_ineffective_count", None)
    if callable(setter):
        try:
            setter(self._session_id, self._ineffective_compression_count)
        except Exception as exc:
            logger.debug("persist ineffective count failed: %s", exc)

3. Load on bind. When the compressor binds to a resumed session, it reads the persisted counter back into memory.

def bind_session_state(self, session_db, session_id):
    self._session_db = session_db
    self._session_id = session_id
    self._ineffective_compression_count = 0  # fallback

    # Load the durable count
    getter = getattr(session_db, "get_compression_ineffective_count", None)
    if callable(getter):
        try:
            stored = getter(session_id)
            self._ineffective_compression_count = max(0, int(stored))
        except Exception:
            pass

That's it. The bug only existed because the codebase had two different patterns — one durable (cooldown, fallback streak) and one in-memory (anti-thrashing counter). The fix wasn't new architecture. It was making the third counter use the same durable channel as the first two.

Why This Works

The key insight is that a session is a durable entity, but its compression state was ephemeral. Every other part of the session — messages, config, metadata — lives in SQLite. The anti-thrashing counter was the odd one out.

By moving it into the same persistent channel, three things happen automatically:

1. Process restarts are transparent. Crash the server, restart it, resume the session — the counter still shows 2, and compression stays blocked.
2. The compression boundary handoff carries the counter. When compression does run (because it's been blocked for long enough that messages grew past the incompressible floor), the rotation from old session to new child session carries the counter. The child starts at the same value the parent had.
3. Concurrent agent access stays coherent. If one agent clears the counter by successfully fitting under the threshold, another agent resuming the same session sees the cleared value immediately.

The reset semantics didn't change, either. Any real provider response that reads below the threshold still clears the counter — and now it clears durably.

Gotchas

Don't persist no-change verdicts. If the counter didn't change — say, a response came in but _verify_compaction_cleared_threshold was False (meaning no compaction happened this cycle) — skip the DB write. Two reasons: it's wasted I/O, and it creates a false positive "someone touched this" signal in the session row's modification time.

Mind the null-coalescing. A session row created before the column existed will return None, not 0. Always max(0, int(stored)) to avoid TypeError on a fresh None.

The guard must check the in-memory value first, then refresh. should_compress() runs on the hot path — every turn start. The in-memory copy covers the fast path. If it says "blocked", use the durable fallback and skip the DB read. Only refresh from the database when the in-memory copy is zero (disarmed) and you need to prove another agent didn't arm it since bind_session_state().

Don't remove the in-memory copy just because the DB is durable. Writing to SQLite on every response is fine (it's a single-row UPDATE on an indexed primary key), but reading from it on should_compress() when you already have the value in RAM is wasteful. Keep both, write through, read from RAM.

What about you?

Have you ever found a bug that was just "this value is in memory but everything else is in the database"? I'd love to hear what patterns you use to prevent state fragmentation in agent systems — especially if you're dealing with context windows and compression. Drop your story in the comments.