Silent Failures, Part 2: When the Code Is Fine but the Ground Is Rotten
When internal architecture is sound but the execution environment silently degrades your agent
TL;DR
Part 1 covered five silent failures inside agent memory architecture — internal degradation. This part covers what lives underneath: the execution environment, the position of memories in the context window, and the structural imbalances that distort what the agent actually sees. The code can be correct. The ground it runs on can still rot.
Introduction
You applied the fixes from Part 1. Weight decay instead of deletion. Summarization instead of truncation. Fresh prompt every request. End-to-end tests before deploy.
The system looks clean.
And yet — something still drifts. Slowly. Inconsistently. Hard to reproduce.
The problem isn't in the code anymore. It's in the ground the code runs on.
Failure 6: Runtime Environment Decay
A system running continuously for weeks without restart accumulates state that was never designed to accumulate.
A real case: One of our deployed assistants began reporting duplicate messages in its context. Diagnostics showed nothing — the database had no duplicates, the retrieval logic was correct, the code hadn't changed. A system restart cleared the problem completely. It never came back — until weeks later.
The cause wasn't in the agent's code. It was in the execution environment. Process-level caches, stale runtime state, artifacts from weeks of continuous operation — all invisible to application-level diagnostics.
Pattern name: runtime decay. The execution environment ages silently. Behaviour that was deterministic becomes probabilistic — not because of bugs, but because of accumulated runtime state.
Symptoms:
→ Errors that disappear on restart
→ Behaviour that can't be reproduced in a clean environment
→ "It worked yesterday"
→ Gradual inconsistencies with no code changes
Why this matters more than it seems:
During normal system maintenance, operating systems, dependencies, and runtime environments evolve through updates. Kernel patches change system call behaviour. Package updates modify shared libraries. Python environments accumulate cached bytecode from previous versions. Each change is minor. Their accumulation over weeks is not.
The environment is the bug.
The correct approach:
First, collect diagnostics while the problem is still reproducible — logs, process state, cache sizes, memory usage. Restarting clears runtime state, but it also destroys the evidence. Diagnose first, then restart.
1. Behaviour drifts without code changes?
→ Collect diagnostics (logs, process state, cache)
→ Then restart before debugging code
2. System updates applied?
→ Restart the agent too
3. Can't reproduce in a clean environment?
→ The environment is the variable, not the code
Important caveat: Restarts clear accumulated runtime state. They do not repair corrupted databases, poisoned embeddings, broken summaries, or invalid records already written to persistent storage. If the problem persists after restart, it's in the data — not in the environment.
Failure 7: Injection Position Blindness
The agent receives its context. All the right memories are there — correctly weighted, deduplicated, fresh.
But where are they positioned in the prompt?
Research on transformer attention — notably Liu et al., "Lost in the Middle: How Language Models Use Long Contexts" (2023) — demonstrates that models process context unevenly. The beginning and end of a context window receive more attention than the middle. Long contexts develop a dead zone — content that exists but is processed with reduced focus.
Context window:
[BEGINNING] ← high attention ✅
[...middle...] ← attention drops 🟡
[...middle...] ← reduced visibility 🔴
[...middle...] ← reduced visibility 🔴
[END] ← high attention ✅
If your most important memory lands in the middle of a long prompt — it's there, but the agent processes it with less focus. This isn't a retrieval failure. It's a positioning failure.
Pattern name: position blindness. The right content in the wrong position loses influence.
The correct fix: position-aware injection.
def inject_memories(memories: list, system_prompt: str) -> str:
# Sort by importance
critical = [m for m in memories if m["importance"] >= 0.8]
standard = [m for m in memories if m["importance"] < 0.8]
# Critical memories: beginning and end of context
# Standard memories: middle (lower attention is acceptable)
half = len(critical) // 2
return (
format_memories(critical[:half]) # top — high attention
+ "\n" + system_prompt + "\n"
+ format_memories(standard) # middle — standard
+ "\n"
+ format_memories(critical[half:]) # bottom — high attention
)
The principle: put what matters where attention lives. This doesn't require complex engineering — just awareness of how the context window is structured.
Note: The severity of this effect depends on the model, context length, and content type. For short contexts (under 4K tokens), position matters less. For longer contexts (16K+), the effect becomes increasingly noticeable. Test with your specific model and context size.
Failure 8: Record Size Imbalance
The memory database contains 200 records. Retrieval selects the top 10 by weight.
One of those 10 is 1800 tokens long.
Total context budget: 4000 tokens
One large record: 1800 tokens (45%)
Remaining 9 records: 2200 tokens (55%)
A single verbose memory consumes nearly half the context budget — crowding out nine other potentially relevant records. The large record might not even be more important. It's just bigger.
Pattern name: size displacement. Record size, not record importance, determines context share.
This happens naturally over time. An agent generates a long analytical response. It gets stored as a memory. Months later, that one verbose record consistently wins context space over dozens of shorter but more relevant memories — simply because it's large.
The correct fix: token-aware budgeting at retrieval. Here's a simple greedy approach:
def retrieve_within_budget(memories, token_budget=2000):
"""Select memories by importance, respecting token budget.
This is a simple greedy approach — select the most important
records that fit within the budget. More sophisticated methods
(knapsack optimization, importance-per-token ranking) are possible
but rarely necessary in practice.
"""
selected = []
used = 0
for memory in sorted(memories, key=lambda m: -m["importance"]):
size = count_tokens(memory["content"])
if used + size <= token_budget:
selected.append(memory)
used += size
return selected
Every record competes on importance, constrained by size. A 1800-token record can still be selected — but only if it's important enough to justify the space it takes from others.
Additionally, consider normalizing record length at write time. When a memory exceeds a reasonable length (the specific threshold depends on your architecture and context budget), summarize it before storage — never truncate. This prevents size displacement from accumulating in the first place.
Tying It Together: Write Quality
Parts 1 and 2 share a common thread that deserves explicit mention: what you write to memory matters as much as how you read it.
Part 1 introduced contextual importance scoring — calculating importance based on content type, not message length. Part 2 adds two more write-time concerns:
Size normalization: Long records stored without summarization cause size displacement at retrieval (Failure 8).
Environmental awareness: Accumulated runtime state can corrupt what gets written, even when the code is correct (Failure 6).
Summarization quality: A poor summary — one that loses key facts, drops names, or flattens emotional context — is itself a source of long-term corruption. The summary replaces the original in future retrievals. If it's inaccurate, every future interaction built on it inherits that inaccuracy. Summarization must preserve meaning, not just reduce length.
The write layer is the first line of defense. If low-quality data enters the database — whether through poor scoring, missing normalization, or broken summaries — no downstream optimization compensates. Clean at the source.
Protecting Context Across Restarts
Restarts are necessary — but unprotected restarts are themselves a source of silent failure. If the agent process is killed without saving state, the current conversation context is lost. The agent resumes with no memory of the last exchange. The user notices. Trust erodes.
The pattern: Two scripts that bookend every restart cycle.
Graceful shutdown saves the last state before stopping:
Stop script:
1. Send SIGTERM (not SIGKILL) → agent catches the signal
2. Agent writes last messages to the database
3. Agent records shutdown timestamp and dialog state
4. Process exits cleanly
5. If still running after timeout → SIGKILL (last resort)
The key detail: the shutdown record itself becomes part of the memory. When the agent restarts, it knows it was stopped — and when. This prevents the disorienting gap where the agent has no explanation for lost time.
Diagnostic startup verifies everything before the first message:
Start script:
1. Check execution environment (conda, dependencies)
2. Check configuration (API keys, paths)
3. Check database accessibility (all databases)
4. Check memory layer availability (each layer independently)
5. Check dialog continuity (active dialog exists, last state recoverable)
6. If any check fails → report the problem, suggest a fix, do NOT start
7. All checks pass → start the agent → script exits
The startup script never crashes. If a database is missing, it says which one and how to recreate it. If a dependency is broken, it says which package to install. It doesn't start a half-functional agent — it either starts a fully verified system or explains what's wrong.
Why this matters for context protection:
Without graceful shutdown, every restart truncates the conversation mid-stream. Without diagnostic startup, the agent may launch with a missing memory layer — and silently operate without it, never knowing what it lost.
Together, these scripts turn restarts from a context-damaging event into a context-preserving one. The agent shuts down knowing it will resume. It starts up knowing everything is intact.
The Complete Diagnostic Checklist
Across both articles, the full set of protections:
Memory Architecture (Part 1 — internal):
□ Weight decay on injection, not deletion
□ Summarization, not truncation
□ Fresh prompt every request
□ End-to-end tests before deploy
□ Diagnostic dialog after architecture changes
□ Monitor injected records per request
Execution Environment (Part 2 — external):
□ Collect diagnostics before restarting
□ Restart agent after system updates
□ When behaviour drifts without code changes — environment first
□ Restarts clear runtime state, not data corruption
□ Graceful shutdown saves last state to database
□ Diagnostic startup verifies all layers before first message
□ Startup script reports problems and suggests fixes, never crashes
Write Layer (both parts):
□ Contextual importance scoring at write time
□ Size normalization at write time
□ Verify summarization quality preserves key facts
Context Structure (Part 2):
□ Position-aware injection (critical content at top and bottom)
□ Token-based budgeting, not record-count budgeting
□ Dead zone awareness for long contexts
Conclusion
Part 1 examined what happens inside the memory architecture when things go wrong — internal degradation that accumulates silently within the system's own logic.
Part 2 examined what happens when the architecture is correct but the execution environment around it isn't — external degradation from runtime state, context positioning, and structural imbalances.
The answer is the same in both cases: silent degradation. The agent keeps working. Just slightly worse. And it won't tell you — unless you ask.
The integrity of the execution environment is not a secondary concern. It is part of the architecture. Maintain both.
Part of the series:
- Memory-Safe AI Development — how to design memory systems
- Silent Failures, Part 1 — internal degradation
- Silent Failures, Part 2 — external and environmental degradation
Author: Aleksandr Kossarev, Jõgeva, Estonia
Tags: #ai #architecture #programming #softwareengineering
Top comments (0)