DEV Community: weiwuji

From Loop to Graph: Our 52-Day Agent Engineering Evolution

weiwuji — Wed, 19 Aug 2026 13:18:16 +0000

The Pain: You just wrapped your head around Loop Engineering, and now the AI world is talking about Graph Engineering. You learn one concept, the next arrives — forever chasing, never landing.
What You'll Learn: A real, working Agent engineering evolution path (Prompt → Context → Loop → Graph), each layer with physical evidence — not invented concepts, but steps I actually took over 52 days.

0. Not a Concept Race, an Engineering Evolution

A recent essay, "Beyond Workflow: Toward Graph Engineering," nailed the key point:

"Many people think the upgrade path for Workflow is adding more nodes, more branches, and more conditionals. But once complexity truly grows, the question is no longer 'how to add nodes' — it's how to describe entities, relationships, state, and dynamic decisions in a changing environment. That is the fundamental divide between Graph Engineering and traditional Workflow."

Many treat Graph Engineering as "yet another concept" to chase. For practitioners like us, it is not a concept — it is the next stop on an evolution path. And I have genuinely walked that path.

1. Scaffold Evolution Overview: Prompt → Context → Loop → Graph

Stage	Core Question	Abstraction Level	Bottleneck
① Prompt Engineering	How to make the model understand	Single conversation	Complex tasks can't finish in one shot
② Context Engineering	How to give enough context	Single session	Limited window / noisy input
③ Loop Engineering	How to execute in loops	Looping execution	Runaway loops / no state
④ Graph Engineering	How to coordinate multiple agents	Multi-agent orchestration	State management / failure localization

Evolution logic: each stage solves the bottleneck of the previous one —

Prompt (single-shot capability) → Context (memory capability) → Loop (execution capability) → Graph (organization capability)

2. Our Context: Query the Index First, Then Feed Context

My first stop was Context Engineering. The classic pitfall: feeding the entire knowledge base to the LLM — context explosion, burning tokens, slow answers.

I switched to a two-step method:

Query the index: the knowledge base maintains a static "concept → article" mapping table plus rule matching, locating "which nodes to look at" in milliseconds
Feed relevant fragments: only the matched nodes go to the LLM — slimmer context, dramatically lower token cost

This is a lightweight RAG: no vector database, no embeddings — cheaper and faster for small knowledge bases.

3. Our Loop: Harness Engineering (the Core of 52 Days)

The second stop was Loop Engineering — my deepest investment and biggest payoff. Its core isn't "let the loop run," it's making every loop iteration stronger:

Error occurs
→ ① error-ledger records it (symptom → root cause → fix → status)
→ ② correction sedimentation (distill into rules/skills)
→ ③ physical rule materialization (write into verify scripts/gates)
→ ④ gate immunity (blocked at the gate)
→ back to ① (new errors keep coming)

Real evidence (what I ran over 52 days):

error-ledger: every corrected error is structurally recorded (symptom → root cause → fix → status)
Daily self-evolution cron (21:00 every night): automatic error review + hot-topic research + topic suggestions
Gate matrix: 6 physical gates before task output (task context exists / skill exists / rules executed / timeout / regression / schema)
Maker/Checker separation: producer and verifier are physically separate — no self-review

This is the complete closed loop from "Self-Improving Agents Are Not a Myth" — reflection is probabilistic, evolution is physical.

4. Our Quasi-Graph: Multi-Agent Collaboration (in Progress)

The third stop is Graph — we're already running it, we just hadn't named it in "graph" language yet.

Using my production business agent system as an example:

Nodes: Email Agent / Report Agent / Ticketing Agent / Writing Agent (each with its own role)
Edges: Maker produces → Checker validates → gate allows/blocks (the collaboration relationship)
Shared state: SQLite (ticketing) + ledgers + knowledge base (append-only)
Governance: rules → verify scripts → gate matrix (the governance layer of the graph)

Key insight: we never wrote a line of "graph framework" code, yet our system is naturally a graph — because as complex tasks multiply, agents naturally evolve from "one person doing everything" to "a team working together". Graph isn't invented; it's the natural result of evolution.

5. The Four-Layer Agent Engineering Scaffold (Practitioner Path)

If you want to build your own Agent engineering system, build it in this order (the order I actually hit):

Layer	What It Is	Verification Standard
① Context Engineering (foundation)	Knowledge base + index + memory management	Knowledge retrieval hit rate > 90%?
② Loop Engineering (skeleton)	Loops + error records + correction sedimentation + gates	The same error never happens twice?
③ Observability (nervous system)	Gates + audit + tracing + ledgers	Can you localize a failure within 5 minutes?
④ Graph Engineering (organization)	Multi-agent + state management + dynamic decisions	Adding an agent doesn't disturb the others?

Don't skip layers: jumping straight to Graph without solid Context = a graph full of "amnesiac nodes." My 52-day order was Context → Loop → quasi-Graph, with evidence at every layer.

6. Cognitive Upgrade: Graph Is the Watershed

Graph Engineering is the watershed from "engineering" to "architecture."

Before (Prompt/Context/Loop): how to make an Agent do work (engineering)
After (Graph/platform): how to design an Agent organization (architecture)

Future AI competition isn't only about model capability — it's about how to design an intelligent system that reliably completes complex tasks. That consensus is growing. And for designing "intelligent systems," the graph is the correct abstraction.

For us: Graph Engineering is not another concept — it's the engineering answer to "how to manage agents once you have many."

7. Where You Are Now

You no longer settle for "learning one more concept" — you start asking "which layer is my agent system on now, and how should it evolve next."

You are becoming the practitioner who turns concepts into scaffolds.

Remember: Prompt asks, Context remembers, Loop runs, Graph organizes. Evolution with physical evidence at every layer is what real engineering looks like.

📖 Further Reading

Self-Improving Agents Are Not a Myth: A Complete Loop from Error-Ledger to Loop Engineering — the full practice of the Loop stage
Practice = Technology × Scenario × Value: What Cognitive Monetization Means in the AI Era — the practitioner methodology
Why "All-Powerful" Agents Fail: Scene Routing That Splits One Agent into 6 Experts — node splitting for multi-agent collaboration
The Observability Trio: Gate + Audit + Correction — A Feedback Loop for Commercial Agents — the governance layer of the graph

About the author: Wu Ji (无记) — AI & digitalization practitioner focused on Agent engineering, Loop Engineering, and digital transformation. Practical, hands-on tutorials — follow along and it just works.

Self-Improving Agents Are Not a Myth: A Complete Loop from Error-Ledger to Loop Engineering

weiwuji — Mon, 17 Aug 2026 15:07:23 +0000

The Pain: The mistake your agent made yesterday, it makes again today. You re-teach it every time — "Remember! Don't do that again!" — and it still forgets. A lot of people say "Self-Improving Agents are a myth," because for most agents, "reflection" is just talk. It never becomes system behavior.
What You'll Learn: A complete, production-proven self-improvement loop for agents (error-ledger → corrective distillation → rule materialization → gate immunity), so that every mistake the agent makes is a lesson the system permanently remembers.

0. The Hottest Direction of 2026 — and the Most Questioned

"Self-Improving Agents" is becoming a top-tier topic:

Stanford launched a course (CS329A) devoted entirely to Self-Improving AI Agents
ICLR 2026 has a dedicated workshop on Recursive Self-Improvement
Google engineer Addy Osmani wrote Self-Improving Coding Agents

But on the other side of the debate, the loudest argument in the community is:

"Self-improving AI agent is a myth"

Why a myth? Because for most agents, "self-improvement" looks like this:

Reflect (the LLM says "I was wrong") → done

No record, no distillation, no verification — the same mistake happens next time. That isn't self-evolution. That's self-comfort.

1. The Essence of Self-Improving: It's Not "Thinking", It's "Doing"

What's the difference between a self-improving agent and an agent that simply "reflects"?

Reflective: the LLM outputs a paragraph of "I'll be careful next time"
Self-improving: mistake → record → distill a rule → block it physically (impossible to repeat)

Reflection is probabilistic (the LLM talks a good game, but may forget next time); evolution is physical (the rule is written into the system, so next time it is guaranteed to be blocked).

That's exactly why people call self-improvement a myth — most people are doing "reflection," not "evolution."

2. error-ledger: The Bookkeeping of Self-Evolution

The first step of the self-improvement loop I run in production is the error-ledger.

Any mistake / failure / correction
  → logged into error-ledger.md (symptom → root cause → fix → status)
  → not just "noted down", but structured recording

Field	Example
Symptom	The new-email report shows only a count, no details
Root cause	Generated mechanically, no per-email analysis
Fix	Every email must show sender + subject + linked ticket ID
Status	embedded (hardened into rules)

Why does structured recording matter? Because an LLM's "memory" is unreliable — context gets compressed and overwritten. But files are physical. They're always there.

3. Corrective Distillation: From "Record" to "Knowledge"

Recording is only the first step. The second step is distilling mistakes into reusable rules:

Errors in the error-ledger
  → scanned daily / weekly
  → extract "the principle behind this error"
  → distill into: skills / SOPs / gates

What I actually do:

Every time I get corrected → log it into the error-ledger → extract the knowledge → write it into a skill (SKILL.md) or a rule
Example: my boss corrected me on "don't dismiss other people's viewpoints" → written into the skill as a "writing perspective iron rule" → every article written since then complies automatically

This step turns a "one-time mistake" into a "permanent asset."

4. Rule Materialization: Making the System "Impossible" to Repeat

The highest form of knowledge isn't the kind written in a document — it's the kind baked into the physical logic of the system:

Rule (a human remembers) → Script (the system executes) → Gate (forced interception)

My three levels of materialization:

Write into skills / docs: followed automatically when the LLM loads them (probabilistic)
Write into verify scripts: validated automatically at check time (deterministic)
Write into gates: output is blocked outright if the gate fails (forced)

Example: before an article goes out, check_cross_links.py validates the cross-links → it fails the build if they're missing → publishing with defects becomes impossible.

This is the watershed between "evolution" and "reflection" — reflection says "I'll try," evolution says "I guarantee."

5. The Complete Loop: Self-Evolution as Loop Engineering

String the four steps together and you get the full self-improvement loop:

A mistake happens
  → ① error-ledger records it (symptom → root cause → fix)
  → ② corrective distillation (distill the principle / rule)
  → ③ rule materialization (scripts / gates)
  → ④ gate immunity (impossible to repeat)
  → back to ① (new mistakes keep flowing in)

Every time the loop turns, the system learns one more lesson. This isn't a myth — it's something I run every day in production.

What's the relationship between this loop and Loop Engineering? Loop Engineering turns an agent's thinking into an observable, controllable loop; self-evolution is what makes that loop stronger with every revolution.

6. Where You Are Now

Right now, you're no longer satisfied with "just letting the agent reflect" — you're starting to think about "how to make the system physically remember its lessons."

You're becoming the kind of engineer who — turns every mistake into system immunity.

Remember: reflection is probability, evolution is physics. Self-Improving Agents are not a myth — they just need the complete loop from error-ledger to Loop Engineering.

🏷️ Entities: Self-Improving Agents, Loop Engineering, error-ledger, Gates
💼 Value: Error immunity, System evolution, Rule materialization, Production-proven
🧠 Cognition: Reflection is probability, evolution is physics — the complete loop from error-ledger to gate immunity

📌 About the author

About the author: Wu Ji (无记) — AI / Agent / digital transformation practitioner. I only write about things I've actually built and run — no concepts without practice. Follow along, and let's turn cognition into income.

📖 Further Reading

Choice Can't Stop at Tools — Two Ways the Content Flywheel Turns · How the flywheel cuts through the value layer
Practice = Technology × Scenario × Value · The complete loop of turning cognition into income
DeepSeek Open-Sourced Harness: Everything Is a Plugin, but the Real Gap Is Local · Foundations converge; the gap is in the details

数智化转型实战·第三篇：从SOP到组织免疫力——让事故变成能力

weiwuji — Sun, 16 Aug 2026 13:14:47 +0000

The Pain: The landing trio (onboarding checklist, checkpoint gates, rollback) lets a new hire take over your system in a week. Then you find a harsher truth: incidents still come, and they love hitting the exact same spot. My real record: Aug 3, three finished articles never entered the publish queue, patched that night; Aug 5, the same gap again - the English versions were translated but publishing and registration were skipped, patched again; Aug 6, the third time - all five Series D articles were live, and the queue still did not list them. The same "finished article not registered" pit, stepped into three times in five days.
What You'll Learn:

Why "knows how to fix" is not "has immunity" - fixing solves this one, immunity solves the next one

The 4-step organizational immunity loop: incident → log → fix → feed rules back

How each step maps to a real artifact: error-ledger entries, re-runnable verification, rules sunk into skills/scripts/gates

How to verify immunity is actually working - two signals you can check with one command

Opening: the same pit, we stepped into it three times

In the previous article - From Documents to Mechanisms: Onboarding Checklist, Checkpoint Gates, and Rollback - the trio got a new hire productive in one week. Then you discover a harsher fact: incidents still come, and they love hitting the exact same spot.

My real record: on Aug 3, during multi-platform publishing, we found that articles 17/19/20 had never entered the publish queue even though they were finished - patched that night. On Aug 5, the same gap again: the English versions were translated, but publishing and registration were not done - patched again. On Aug 6, the third time: all five Series D articles were already live, and the queue still had no trace of them. The same "finished article not registered" pit, five days, three times.

The team is not lazy; the system has no immunity. One person stepping into a pit is a lesson; an organization stepping into the same pit repeatedly is amnesia. Today's cure is a four-step loop: incident → log → fix → feed rules back. This loop is not a concept; it is a component running in my own system right now, and every command can be reproduced on your machine.

Why "knows how to fix" is not "has immunity"

Most teams' "review" looks like this: incident happens → the person in charge fixes it overnight → a message goes into the group chat: "fixed, be careful next time" → three months later the same pit explodes again. Why? Because fixing solves "this time"; immunity solves "next time".

Gap	What it looks like	Consequence
No logging	Fixed and done, no record	The same troubleshooting starts from zero again
No feedback	The record lies in a document	The rule never reaches the execution layer
No interception	The rule depends on people remembering it	Newcomers do not know the old pits

The essential difference: fixing puts the system "back to normal"; immunity makes the system "unlikely to break in the future". To achieve the latter, every incident must become an increment of the rules - this is what Loop Engineering looks like at the organizational level.

Step 1: Log the incident - turn "pits you stepped into" into "organizational assets"

Problem: incident memory lives in individual heads. The person involved remembers it clearly, but the organization has no memory. The person leaves, the lesson leaves with them.

Solution: create an error ledger and log in four parts. First, create the error knowledge base file /root/shared/error-ledger.md; second, every incident gets one entry with a fixed four-part structure: symptom → root cause → fix → status. Our knowledge base has accumulated more than 30 real incidents. Here is a real entry (the 2026-08-01 Hashnode duplicate-publish incident):

## 2026-08-01 · Hashnode duplicate publish incident
**Symptom:** A1/A2/A3/A4/A5/A8 each appeared twice on Hashnode
**Root cause:** the "Draft not found" error actually meant the publish had
        succeeded; the retry logic caused duplicates
**Fix:** dedupe before publishing - skip when the title already exists;
        when "Draft not found", first check whether the article is published
**Status:** dedupe logic added to multi_publish.py

The four-part structure is not a format; it forces you to think clearly: symptom states "what you saw", root cause digs to the mechanism layer, fix goes down to command level, and status states clearly whether the loop is closed. A pit that is not closed is written in the open, not hidden in your head.

✅ Verification: whenever you hit a new error, the first step is not to fix it - it is to open the error ledger and search history. Run this command:

grep -n "duplicate\|429\|Broken pipe" /root/shared/error-ledger.md

10 seconds to a historical answer, no need to troubleshoot from zero.

🩸 Pitfall: on Aug 2, the daily report trend date showed question marks; after troubleshooting we found the collected_at field was missing - but this pit had never been logged, so the lesson was wasted. The rule now: log today's incident today, do not wait for the review meeting.

▸ Cognitive shift: logging is essentially turning "pits you stepped into" from individual memory into organizational assets - assets appreciate, memories depreciate.

Step 2: Verify the fix - "fixed" is not the end, gate passed is

Problem: announcing "fixed" right after fixing is the fastest path to recurrence. Whether a fix actually worked cannot rely on feeling; it relies on evidence.

Solution: every fix must come with verification, and the verification must be re-runnable. When we handled the "gate_en_articles.py Chinese detection false positive" incident, the fix was changing the detection logic (character count → line count), and the verification was running three regression cases (compliant article passes / Chinese residue blocked / passes after the fix):

# After the fix, run the verification; 4/4 PASS is the real closure
python3 /root/hermes-harness/scripts/validate_article.py check article-25.md

Real output (I ran it on my machine):

check: article-25.md | PASS
body: 3666 chars
quality score: 100/100 (16/16 checks passed)
ALL CHECKS PASSED - article meets the publish standard

🩸 Pitfall: on Aug 5, fixing the Dev.to tag error (the tag contained a hyphen ab-testing, and the platform only allows alphanumeric characters - fixed to abtesting). Right after fixing, we re-ran the publish and confirmed the 422 was gone, and only then changed the status in the error ledger to "fixed". Verification is not optional; it is the switch that closes the loop.

▸ Cognitive shift: verification is essentially turning "I fixed it" from a self-report into a set of re-runnable evidence - evidence speaks, self-reports lie.

Step 3: Feed the rules back - stop the next incident before it happens

Problem: logged, fixed - and the rule still lies in the document. The next incident happens anyway. The key step of an immune system is to sink rules from the "knowledge layer" down to the "execution layer" - into skills, into scripts, into gates.

Our real feedback actions, three paths:

1. Sink into the skill: on Jul 31, the WeChat push returned 501 (consecutive SQL keywords in the body triggered security risk control); the fix was to express such patterns as pseudocode/field descriptions, fed back into the content-repurpose skill, so future articles avoid it automatically.

2. Sink into the script: on Aug 1, the Dev.to 429 rate limit (about 5 posts per 30-second window); the fix was publish interval >= 30 seconds and <= 3 posts per batch, fed back into multi_publish.py with an automatic sleep; the same day's Hashnode duplicate-publish incident got its dedupe logic written directly into the publish script.

3. Sink into the config: on Aug 1, we found the legacy publish CLI was broken (only accepts JSON, depends on a local service, cover URL upload returns 40113); the fix was switching the publish path; on Aug 8, the old command and plaintext credentials were removed from the cron task together, fed back into the task config, never misused again.

# Real effect after feedback: the publish script auto-dedupes and auto-rate-limits
# same title already exists -> skip; between publishes -> auto sleep 30s
grep -n "dedupe\|sleep\|30" /root/tools/baoyu-post-to-wechat/scripts/wechat-api.ts | head -5

🩸 Pitfall: feedback is the easiest step to skip. In the Aug 3 queue gap, the fix was "re-publish", but the feedback action (registering the queue right after writing) was not done at the time - so Aug 5 and Aug 6 it happened again, back to back. A fix without feedback is burying the pit again.

▸ Cognitive shift: feedback is essentially turning "be careful next time" from a verbal reminder into a line of checking code - once the rule enters the execution layer, it does not depend on anyone remembering it.

Step 4: Verify immunity - the same pit does not recur, and if it recurs you can look it up in seconds

Problem: after the feedback, how do you know immunity actually works?

Solution: two signals. First, the same error no longer recurs the same way; second, even if it recurs, you can find the historical answer within 10 seconds.

Our cron task crashed three days in a row (Aug 4/7/8, dying at startup). The first diagnosis was "one-night infrastructure failure"; after the Aug 7 recurrence it was corrected to "recurrent occasional failure"; after the third time on Aug 8 it was upgraded to "persistent occasional failure" and physical governance began (removing the broken command, updating the task config). This diagnosis-correction chain itself is the immune system at work - every recurrence makes the understanding more accurate.

Another real case: after the queue gap recurred three times (Aug 3/5/6), we hardened "register the queue immediately when writing is done" into the process, and that class of gap has not appeared since.

✅ Verification: one command tells you whether immunity is working:

grep -c "fixed" /root/shared/error-ledger.md    # fixed entries keep growing
grep -n "queue gap" /root/shared/error-ledger.md # when did it last recur

▸ Cognitive shift: verifying immunity is essentially turning "we will not do it again" from an optimistic expectation into checkable history - the thickness of the error ledger is the scale of organizational immunity.

Before vs After: the same incident, two endings

Stage	Before: individual debugging	After: organizational immunity
Incident record	Fixed and forgotten, "fixed" in the group chat	Logged in four parts the same day, everyone can look it up
Troubleshooting cost	The same error investigated from scratch	grep gives the historical answer in 10 seconds
Where the rule lives	In the lead's head	In skills / scripts / gates
Recurrence rate	The same class of pit stepped into repeatedly	Rules intercept, the same pit does not recur
Newcomer onboarding	Step into every pit from scratch	The error ledger is a pit-avoidance map

Why an immune system beats a perfect system

A lot of teams chase "never making mistakes". Wrong direction. Chasing never making mistakes only makes mistakes get hidden; chasing immunity makes mistakes become nutrition.

The immune system has three properties, and they map exactly to the three things we built: memory - the error ledger makes every incident remembered; learning - rule feedback makes every fix change system behavior; detection - gates and verification make recurrence discoverable. Remove any one of the three and it is not immunity, it is just "fixing a bit faster".

Go one layer deeper: the immune system solves the "survival problem" of organizational knowledge. When individuals debug, knowledge lives in human heads - the person leaves, the knowledge leaves. When organizations immunize, knowledge lives in mechanisms - as long as the mechanism is there, the knowledge is there. The third real question of digital transformation is not "make the system smarter"; it is letting the organization evolve from "knows how to fix" to "does not break" - growing a memory every time it makes a mistake. This is the complete shape of Loop Engineering at the organizational level.

Summary: 3 things you can start today

Create an error ledger: four parts (symptom/root cause/fix/status), log today's incident today
Every fix must come with verification: after fixing, run a re-runnable verification command; only pass means change the status
Force the feedback: after every fix, sink the rule into a skill, a script, or a gate - pick one of the three

Do these three and your organization starts to have immunity. This is the third foundation stone of digital transformation - from "knows how to fix" to "does not break", from "individuals remember" to "mechanisms remember".

Next article: LLM-as-Judge: How to Calibrate Your Judge

After rules are fed back into the system, a new question appears: if the judge deciding "right or wrong" is itself an AI, who verifies that the judge is not blind? Next we enter the Agent evaluation and observability series, and talk about calibrating the AI judge - the last link of organizational immunity: even "judgment" itself must be examined.

🧭 Cognitive Index

🏷️ Entities: error-ledger · multi_publish.py · content-repurpose · four-part logging
💼 Value: organizational immunity · incident closure · knowledge consolidation
🧠 Cognition: from "knows how to fix" to "does not break" - the thickness of the error ledger is the scale of immunity

About the author: Wu Ji (无记) — AI / Agent / digital transformation practitioner. Only writes things that actually ran end to end, no concepts for their own sake. Follow along and it just works.

In Diagrams, Precision > Pretty: Why AI Technical Articles Use Code-Generated Images

weiwuji — Fri, 14 Aug 2026 08:18:14 +0000

The Pain: When adding diagrams to technical articles, there are two classic pitfalls — either you paste dense screenshots that readers can't make out, or you have AI draw a lovely illustration that nobody can tell what it's about.
What You'll Learn: An understanding of the principle "precision > pretty" in diagrams, and why code-generated images are the best choice for technical articles — precise as the baseline, and good-looking on top of it.

0. The essence of diagrams: making the point clear

Diagrams in technical articles aren't decoration — they're information carriers.

A good technical diagram: one glance and the reader grasps "how this flow works," "where this comparison differs," "how many layers this architecture has."

A bad technical diagram: beautiful, but the reader leaves more confused than before.

1. Two ways to make diagrams

Way A: AI text-to-image (let the model draw)

Input: a description → Output: an AI-generated image
Pros: great visual texture (3D materials/magazine feel)
Risks: ⚠️ Text can be wrong/blurry/cropped (AI-drawn text is uncontrollable)
      ⚠️ Data can be mangled (chart semantics redrawn)

Way B: Code-generated images (SVG rendering)

Input: structured data → code generates SVG → renders to PNG
Pros: ✅ Text 100% precise (code-rendered)
     ✅ Data 100% precise (values fully controlled)
     ✅ Free + fully reproducible
Risks: ⚠️ Visual quality requires design work (template-driven)

2. Why "precision > pretty"

What's special about technical articles

General article diagrams: pretty is enough (emotion/atmosphere)
Technical article diagrams: precision is the baseline (information/logic)

Wrong process label → readers follow it and get it wrong
Wrong data on a chart → readers believe it, and everything built on it is wrong

One wrong diagram in a technical article can destroy the trust of the entire piece.

An example

Say you're diagramming an "agent workflow":

Text-to-image: the AI might render "User Input" as "System Input," the label blurs, and readers have to guess
Code-generated: labels like "User Input," "AI Execution," "Result Check" are pixel-perfect

3. My approach: precision first, aesthetics second

My diagram system: code-generated (SVG) as the backbone, borrowing the visual texture of AI imagery.

SVG code generation (guarantees precision)
  + material-style 3D look (borrows the pretty)
  + short labels (2-5 characters, only labels inside the image)
  + QA checks (labels legible / data correct / no cropping)
  = diagrams that are both precise and good-looking

Concretely:

Precision: SVG code rendering — text/data 100% under control
Aesthetics: material-style visuals (soft shadows / IKB blue / Swiss layout) — learn the "texture" from AI text-to-image style, skip its "uncontrollability"
Short labels: in-image text ≤ 6 characters — "Result Check," not "Check the Execution Result"
QA: after generation, check labels / data / cropping

4. Why I don't use AI text-to-image for in-body diagrams

To be clear: AI text-to-image isn't bad — it's great for covers, banners, and decorative illustrations.

My choice is division of labor by purpose:

Purpose	Approach	Reason
In-body concept/flow diagrams	Code-generated (SVG)	Text/data must be precise
Covers/Banners	AI text-to-image	Decorative, no text requirements
Real screenshots/command output	Actual captures	The iron rule of authenticity

5. Where you are now

Right now, you've stopped paying for "a pretty image" — you've started paying for "an image that explains itself."

You're becoming the kind of — strict reader who looks at a diagram and asks "does it explain the point?" before "does it look good?"

Remember: precision > pretty. A technical article's diagrams are information carriers, not decorations — but on a precise foundation, they can absolutely be prettier.

🏷️ Substance: SVG diagrams, code generation, AI text-to-image, short labels
💼 Value: accurate information, trust-building, visual texture, diagram efficiency
🧠 Cognition: Precision > pretty in diagrams — a diagram is an information carrier, not a decoration, but on a precise foundation it can be prettier

📌 About the author
About the author: Wu Ji (无记) — AI / Agent / digital transformation practitioner. I only write about things I've actually built and run — no concepts without practice. Follow along, and let's turn cognition into income.

📖 Further Reading

In Diagrams, Precision > Pretty · This article
Automate Your Blog Images · Early practice of the diagram system
The One-Person Editorial Department · The full content production system

The Choice Can't Stop at Tools: Agent Content Flywheel, Two Ways to Spin

weiwuji — Fri, 14 Aug 2026 08:17:30 +0000

The Pain: You've produced a lot of content and built up some influence, yet you keep feeling "the flywheel that's spinning is someone else's, not mine." Why? It may not be a content-quality problem — your flywheel may simply be stuck at the tool layer.
What You'll Learn: A clear view of the two ways to spin a content flywheel, and why "the choice can't stop at tools" — tools are means, systems are assets, value creation is the endgame.

0. First, see clearly: what a flywheel is

The flywheel effect: content brings influence, and influence brings more content — each turn charges up the next.

But here's the key question: what spins your flywheel?

1. Two ways to spin the flywheel

Spin A: the tool-layer flywheel

I build a useful tool
  → people use it (influence)
  → I become known
  → I sell more tools
  → more content material

This flywheel's traits:

The output is a tool (Skill/code/templates)
The content type is tool tutorials
The moat is the tool ecosystem

Spin B: the value-layer flywheel

Practice (real business runs successfully)
  → Systematization (SOPs/methodologies/architectures, codified and replicable)
  → Influence (expert personal brand)
  → Business value creation (real application scenarios)
  → more practice material

This flywheel's traits:

The output is a system (methodologies/architectures, not a single tool)
The content type is hands-on practice + cognitive upgrade
The moat is methodology + experience (hard to copy)

2. Why "the choice can't stop at tools"

The tool-layer flywheel isn't wrong — it's real, it works, it's proven. But the choice can't stop at tools, because:

1. Tools get replaced

A tool that's great today may have a better one tomorrow. But methodologies don't go stale — the ability to "systematize a complex system" stays valid no matter which tools you use.

2. A tool is a single point; a system is a network

One tool solves one problem; one system solves a class of problems. A system can be replicated into any scenario — individuals can use it, teams can use it, organizations can use it.

3. Tools build "knowing how to use"; systems build "understanding"

Reader after a tool tutorial: "oh, this tool can be used like that."
Reader after a system methodology: "I understand the essence of this problem — I can design it myself now."

3. How to spin the value-layer flywheel (my practice)

My approach: every piece of content gets distilled into part of a system.

One practice article
  → extract a reusable SOP/methodology
  → codify it into a system (architecture/process/checklist)
  → reuse it directly next time a similar problem comes up
  → the system becomes more complete with every turn

This isn't "making content" — it's "building a system with content." Every article is one brick in the system.

4. The two flywheels aren't opposites

Let me be clear: I'm not saying the tool-layer flywheel is bad — for many creators it's more than enough, and it's highly efficient.

What I'm saying is: if your goal is long-term value creation, the choice can't stop at tools.

Tool-layer flywheel: fast, but low ceiling (selling tools/subscriptions)
Value-layer flywheel: slow, but high ceiling (consulting/training/products/system licensing)

You can spin the tool-layer flywheel first to build influence, then upgrade to the value-layer flywheel — from "I built something useful" to "I built a replicable system."

5. Where you are now

Right now, you're no longer satisfied with "making content" — you're starting to ask "what does this content ultimately settle into."

You're becoming the kind of — long-termist who, before publishing anything, asks "can this become part of a system?"

Remember: tools are means, systems are assets, value creation is the endgame. The choice can't stop at tools.

🏷️ Substance: content flywheel, systematization, methodology, tool ecosystem
💼 Value: value creation, replicable systems, long-term assets, influence
🧠 Cognition: The choice can't stop at tools — tools are means, systems are assets, value creation is the endgame

📖 Further Reading

The Choice Can't Stop at Tools · This article
Practice = Technology × Scenario × Value · The complete loop of cognitive monetization
Correction Sedimentation · The core mechanism of systematization

Practice = Technology x Scenario x Value: What Cognitive Monetization Means in the AI Era

weiwuji — Fri, 14 Aug 2026 08:17:22 +0000

The Pain: You've learned a lot of technology — AI, agents, large models — yet you keep feeling "I've learned so much, but I can't put it to use." Why? Because you've mistaken "practice" for "writing code." Real practice is technology meeting scenarios and creating value — a closed loop of all three.
What You'll Learn: The complete formula "practice = technology × scenario × value," and why this is the true path to monetizing cognition.

0. First, let's align on a concept: what "practice" is NOT

Many people think "practice" means getting your hands dirty — running a demo, training a model, standing up a system.

That's "tinkering," not "practice."

Tinkering (you did it)      → Technology layer: I got it running
Practice (you made it work) → Value layer: the technology created business value in a real scenario

Where's the difference? Tinkering means "I know how to use it"; practice means "I used it to solve a problem."

1. Practice = Technology × Scenario × Value (all three are non-negotiable)

Here's my definition of practice:

Technology must meet real scenarios and create genuine business and scenario value — only then can you monetize cognition.

Broken down, it's three indispensable elements:

Element	What it is	Without it
Technology	AI / Agents / digital-intelligence transformation (real substance, not concepts)	No technology = empty talk about scenarios
Scenario	Real business scenarios / production environments / actual projects	No scenario = repackaged popular science
Value	Business value + scenario value	No value = wasted effort

Why "×" and not "+"? Because the three multiply — if any one of them is zero, the result is zero.

Technology(1) × Scenario(1) × Value(0) = 0  ← you got it running, but created no value
Technology(1) × Scenario(0) × Value(1) = 0  ← great idea, but never landed in a scenario
Technology(0) × Scenario(1) × Value(1) = 0  ← knows the business, but not the technology

2. Why "× Scenario" is the dividing line

Scenario is the bridge between technology and value.

Take the same agent technology:

Without a scenario → run a demo, post a tech note, readers walk away thinking "oh, it runs"
With a scenario → dropped into a real business process, solving a real problem, readers walk away thinking "I want to use it like this too"

Scenario turns a "technology demo" into a "business solution."

From my own experience: the same agent-orchestration technique, explained on its own, is "yet another tutorial." Put it in a real business scenario — what it handles every day, how much manual work it saves, what pitfalls we hit — and it instantly becomes "a replicable methodology." That's the multiplier effect of scenario.

3. Why "× Value" is the finish line

Value is the acceptance criterion of practice.

Getting a technology to run isn't the end. You have to ask: what value did this technology create, in what scenario?

How much time did it save?
How much cost did it cut?
How much efficiency did it gain?
What problem did it solve that couldn't be solved before?

A problem without value isn't worth practicing; a practice without value isn't worth writing about.

4. Cognitive monetization: the inevitable result of the closed loop

Once technology × scenario × value closes the loop, cognitive monetization becomes a natural outcome:

Technology (what you can do) → Scenario (where you apply it) → Value (what you create)
        ↓                          ↓
Cognitive upgrade (readers learn from your articles)    Practice landing (readers follow your playbook)
        ↓                          ↓
            Cognitive monetization (trust + paying users + influence)

Why do readers trust you? Because you're not talking concepts — you're talking about real substance that has created value in real scenarios. That's the moat of "practitioner cognitive upgrade" — competitors talk concepts; we deliver substance. Competitors talk technology; we talk scenario value.

5. Where you are now

Right now, you're no longer satisfied by the thrill of "getting a demo to run."

You're becoming the kind of — strict practitioner who, before learning any technology, asks "what value can it create in what scenario."

Remember: technology is the means, scenario is the bridge, value is the destination. Only when all three close the loop is it real cognitive monetization.

🏷️ Substance: technology practice, scenario application, value creation, cognitive monetization
💼 Value: business value, scenario value, practice methodology, cognitive upgrade
🧠 Cognition: Practice = Technology × Scenario × Value — only when all three close the loop is it real cognitive monetization

📖 Further Reading

Practice = Technology × Scenario × Value · This article
The Choice Can't Stop at Tools · The flywheel that penetrates the value layer
The 4-Layer Architecture of a One-Person OPC · The practical shape of this system

DeepSeek Open-Sourced Its Harness: Everything Is a Plugin, But the Real Gap Is in the Details

weiwuji — Fri, 14 Aug 2026 01:11:21 +0000

The Pain: Today everyone is talking about "DeepSeek open-sourced its Harness" and "everything is a plugin" — and those discussions are valuable. But beyond the buzz, I want to share another angle: a plugin system gives you differentiation space, not success rates. What actually separates agents are the less visible details.
What You'll Learn: What DeepSeek Harness is really worth and where its boundaries are — seen from a production system running in real business scenarios. What to steal, and what's actually your moat.

Let's Align on Facts: What Happened Today

On August 13, 2026, DeepSeek open-sourced its Harness. v0.1 developer preview, MIT license, full source code.

One sentence summarizes its architecture philosophy: everything is a plugin.

Models, tools, skills, sessions, sandbox, storage,
Agent Loop, orchestration, UI
        ↓
all replaceable (Cordis plugin system)

Note: this isn't "you can swap a search tool." Even how the agent loops, how it spawns sub-agents, how it saves sessions — all replaceable. The whole runtime is a Lego base.

Four run modes = four plugin combinations:

Mode	One-liner	Who
Standard	Full tool suite	Regular agent tasks
PTC	Model writes code to compose tool calls	Complex tasks
Minimal	Only shell + file edit	Testing models in minimal env
Creation	Agent inspects runtime, tries plugins, composes new modes	🔥 Harness developers

Plus two core designs:

Append-only session log: every trace flows into one event stream
Multi-agent built-in: Spawn/Fork/Pipeline/Ralph Loop

You don't need to memorize these terms. Just remember one sentence — DeepSeek turned "how an agent works" into a plugin system you can freely assemble.

Everyone's Talking About "Everything Is a Plugin." Let Me Share Another View.

First, let me be clear: "everything is a plugin" is genuinely good news — for the first time, an agent runtime can be freely assembled, and that's a real liberation for developers.

But beyond that angle, I want to explore a different view — after architecture converges, where does the real gap actually lie?

Architecture convergence is inevitable. When everyone can plugin-ify tools, loops, and orchestration, those stop being moats. Spawn, Fork, Pipeline, Ralph Loop all have mature precedents — DeepSeek's real innovation is making them config-replaceable plugins.

An industry expert told InfoQ:

Looking at the full harness pipeline, tool calling, memory management, and task planning directions are basically settled. The real innovation will happen in the local details.

He named four:

Memory compression: accumulated memory needs compressing
Conflict cleanup: conflicting memories need organizing
Path reuse: similar tasks reuse existing planning paths
Plan validation: post-generation "compile-style" checks (complete error branches? executable ops? steps within permissions?)

This is where the real gap is. Plugin systems provide differentiation space, not success rates.

My Practice: I've Actually Run All Four

Not armchair theory. I've run all four in real business scenarios — just not on DeepSeek Harness, on our own production system.

1. Memory Compression → My Cognition Distillery

Uncompressed memory drowns the agent.

My approach is "cognition distillation": after every completed piece of work, I don't keep a log — I distill into nodes (problem + insight + practice + links) into a cognition library.

Published articles → distilled into nodes → cognition library (28 nodes)
  ↓
Writing selects precise nodes → articles stay focused

That's memory compression in practice — not deleting memory, but turning it into a higher-density form. DeepSeek confirms this direction.

2. Conflict Cleanup → My Correction Flywheel

The classic symptom of memory conflict: rules fighting each other, agent doesn't know which to obey.

My approach is "correction sedimentation": every error → error-ledger → extract lesson → sediment as rule → gate intercepts.

Production → audit → errors logged (31 entries)
  → rule sedimentation → gate interception → flywheel spins

The key to conflict cleanup isn't cleanup itself — it's building an error-to-rule conversion pipeline so conflicts surface and resolve automatically.

3. Path Reuse → My Scenario Routing

Agents shouldn't plan from scratch every time. Similar tasks follow similar paths.

My approach is "scenario routing": different scenarios → different agent configs/tool sets, reusing proven paths.

Scenario detection → route to matching agent
  → reuse proven path (no replanning from scratch)

4. Plan Validation → My Physical Pipeline Gates

No validation after planning = letting the agent run naked.

My approach is "physical pipeline": every task output must pass gate verification before delivery.

validate_article → check_series → article_checker → publish_gate
        ↓
4 gates, all physical, no LLM self-discipline

What to Steal: Three Things DeepSeek Harness Does Better

Despite the gap being in details, DeepSeek Harness does three things better than us. Worth stealing:

1️⃣ Append-Only Unified Event Stream (Steal This First)

DeepSeek: everything the model sees → one append-only log
  system prompt / reasoning / tool calls / sub-agent orchestration / context injection
Value: shared observability ledger, session forking, replayable evals

Our observability trio (Gate/Audit/Correction) stores separately; their unified event stream is more thorough — all traces in one stream, evals/debugging/replay all based on it.

This directly connects to Trajectory Evals — the evaluator eats traces, and a unified stream makes traces more complete and replayable.

2️⃣ Tool-Call Pipeline

DeepSeek: tool call = request → Hook → approval → permissions → sandbox → timeout → execute → rewrite → log → UI

Our tool whitelist is a static check; theirs is a pluggable pipeline — every stage independently replaceable/composable. Security goes from "switch" to "pipeline."

3️⃣ Creation Mode Thinking

DeepSeek: agent can inspect runtime, try plugins, compose new run modes
Meaning: the harness config itself becomes an object the agent can operate on

This is "nurture-style AI" engineering — letting the agent participate in its own config. We already have correction sedimentation; we can go further: let the agent read/modify its own rule files through tools (we have this capability, can systematize it).

What NOT to Steal: DeepSeek Harness's Boundaries

Cold water again. Three problems:

"Everything replaceable" ≠ higher success rates — plugins give differentiation space; the final effect needs high-quality default plugins, stable composition patterns, and credible eval results
v0.1 migration cost is high — core plugins and interfaces will change fast; entering the ecosystem now means absorbing high migration cost
Multi-agent has no paradigm breakthrough — it's a pluggable hierarchical Supervisor-Worker, far from true Swarm (autonomous discovery/negotiation/competition/dynamic takeover)

Conclusion: you can swap the base. You can't stop the flywheel.

Where You Are Now

You're no longer the developer who gets excited every time "everything is a plugin" trends.

You're becoming the strict engineer who — sees architecture convergence, watches the local gaps, and validates everything in real business scenarios.

DeepSeek solves "how to build." We prove "how to use" with a production system.

Remember: the Harness is the base; the production system is the answer. Plugin systems provide differentiation space. What actually decides success — memory compression, conflict cleanup, path reuse, plan validation — only real business scenarios can force you to build.

DeepSeek Open-Sourced Its Harness (Merged)

weiwuji — Thu, 13 Aug 2026 14:28:31 +0000

⚠️ This article has been merged into the updated version: DeepSeek Open-Sourced Its Harness: Everything Is a Plugin, But the Real Gap Is in the Details

Merged content removed.

From Documents to Mechanisms: Onboarding AI Agents with SOP + Checkpoints

weiwuji — Thu, 13 Aug 2026 13:19:00 +0000

The Pain: Your methodology is materialized. Rules layer, facts layer, pipeline layer, memory layer - the files all exist. Then you hit a new problem: the files are written, but does anyone actually read them? You drop the docs into the shared drive, and three days later you ask the new hire "did you read it?" He says yes. Then he submits work that contradicts the rules in the documents. You check the read logs - the file was opened 3 times, 2 minutes each. Can anyone read a 200-page SOP in 2 minutes?
What You'll Learn:

Why "documents written" is not "documents used" - the three gaps that kill every SOP

The landing trio: onboarding checklist, checkpoint gates, and a rollback mechanism

How each piece maps to a real artifact you can build and verify on your own machine today

Why this trio beats "more training" - and how it makes trust scalable

Opening: 200 pages of SOP, and the new hire still cannot do the job

In the previous article - From Individual Methodology to Team Assets: The 4-Step Materialization Framework - we turned the methodology into four layers: Rule Layer, Fact Layer, Pipeline Layer, Memory Layer. The files are all written: STANDING.md, value practice cards, four physical gates, error-ledger. Then you discover a new problem: the files are written, but will the team actually read them?

Real scenario: you put the documents into the shared drive. Three days later you ask the new hire "did you read it?" He says yes. Then he submits output that contradicts every rule in the documents. You open the read log - opened 3 times, 2 minutes each. Can a 200-page SOP be read in 2 minutes?

Between "documents written" and "documents used" lies an entire mechanism. Most organizational SOPs die in the same place: the file lies in the shared drive, and nobody ever finishes reading it. Today's cure is the landing trio: onboarding checklist, checkpoint gates, rollback mechanism - turning "did you read it" from a verbal promise into a machine-verifiable gate. These three are not invented for this article; they are components running on my own system, and every command can be reproduced on your machine.

Why "documents written" is not "documents used" - the three gaps

Documents go unused not because people are lazy, but because of three gaps:

Gap	What it looks like	Root cause
No path	The new hire does not know which file to read first	"What to learn" is guesswork
No verification	Nobody can confirm whether it was read	"Did you learn it" relies on asking
No safety net	The new hire is afraid to touch anything, scared of breaking it	"Cost of trying" is unknown

So SOP rollout is not about more training. It is about filling three holes: the checklist gives the path, the checkpoint gives verification, the rollback gives safety. Let's open each one.

Step 1: The onboarding checklist - turn "what to learn" into "tasks that must be checked off"

Problem: a document is not a task. Anything without a deadline and an acceptance criterion sits at the bottom of every to-do list forever. The new hire is not unwilling to learn; he simply does not know where to start.

Solution: an onboarding checklist. Every item is "checkable + produces an artifact + verifiable". This is the checklist running in my own system (/root/shared/onboarding-checklist.md), first three days:

## Day 1: read the rules
- [ ] Read STANDING.md end to end, write out any one rule verbatim
      (acceptance: cat /root/shared/STANDING.md | head -30)
- [ ] Read .hermes.md, restate the publish path
      (acceptance: can state that the legacy publish CLI is banned,
       publishing goes through baoyu-post-to-wechat)
## Day 2: run the pipeline
- [ ] Run the 4 gates on article-23.md, screenshot the output
      (acceptance: validate_article.py check returns 100/100)
- [ ] Read 2 value practice cards, restate the "pitfall and fix" fields
## Day 3: produce independently
- [ ] Walk through the full publish flow, confirm the draft appears
      (acceptance: draft/batchget returns the title and content is not truncated)

The key design is the acceptance command: the artifact of every task must be verifiable by one command, not by "I feel like I learned it". Every item on the checklist eventually flows into a real file. Our task board (/root/shared/writer-task-board.md) is a living handover list - one agent writes TODO items, another picks them up and updates the status; tasks move through the file, not through anyone's memory. Account handover uses the same mechanism: the handoff guide ships in two waves (four domestic platforms first, three overseas platforms second), and every item states exactly what needs to be provided.

✅ Verification: once the new hire checks off the list, asking "did you read it" is pointless - the artifacts are sitting there, open the file and check.

🩸 Pitfall: the worst checklist is a "reading list" - "recommended reading: XXX". That is not a checklist, that is a bibliography. A bibliography has no acceptance criteria, cannot be checked off, and becomes the second file gathering dust in the shared drive.

💼 Direct value: the first step of a one-week handover is not "learning fast", it is "path clarity". On day one the new hire knows what to read first, what to produce, and how to verify it.

▸ Cognitive shift: a checklist is essentially turning "what to learn" from an individual judgment into the team's default path - once the path is written down, it no longer depends on who is doing the mentoring.

Step 2: The checkpoint gate - turn "did you read it" into "is the evidence there"

Problem: a checked box can still be a lie. "I read it" is an attitude, and attitudes cannot be verified. You ask the new hire "do you understand?" He says yes. Do you believe him or not?

Solution: the checkpoint does not verify attitude, it verifies evidence. The first comment in our gate script says: never read the LLM's self-report, only read real file evidence. That comment was earned by walking into the pitfall - early on we relied on the agent reporting "done", and its self-reports frequently disagreed with actual state; once we switched to file evidence only, false reports died automatically.

The real artifact is gate-check.sh (under /root/hermes-harness/scripts/hooks/). It does not check "did you read STANDING.md"; it checks "is the artifact that STANDING injection produces present" and "is the task-context file created". Evidence missing → blocked; blocked 3 times → escalated to a human. Zero subjective judgment anywhere in the loop.

The same logic runs harder in the content pipeline: 4 physical gates, and any gate failure means sys.exit(1) - the publish command never executes:

python3 /root/hermes-harness/scripts/validate_article.py check article-23.md
python3 /root/hermes-harness/scripts/check_series_continuity.py check
python3 /root/hermes-harness/scripts/article_checker.py article-23.md
python3 /root/hermes-harness/gate/publish_gate.py article-23.md

Real output (I ran the first gate on this very article on my machine):

check: article-24.md | PASS
body: 3666 chars
quality score: 100/100 (16/16 checks passed)
ALL CHECKS PASSED - article meets the publish standard

✅ Verification: any output only counts as qualified after passing every gate - it does not matter how senior you are or who you are; only evidence matters.

🩸 Pitfall: checkpoints must check "evidence", never "reports". We actually walked into this: the agent self-reported "validation done" while the artifact was missing fields; only the gate scan caught it. Human review has the same flaw - you catch things when you are in a good state and miss them when you are not, so the gate has to be physically enforced.

💼 Direct value: quality moves from "I trust you" to "I verify you". The new hire does not need to prove himself; he only needs to produce evidence.

▸ Cognitive shift: a checkpoint is essentially turning "trust" from a judgment about a person into a check on evidence - trust becomes scalable because it no longer depends on character.

does evidence exist? yes -> next step; no -> blocked, retry, 3 blocks in a row -> escalate to human" width="800" height="587">

Step 3: The rollback mechanism - the confidence that lets new hires dare to try

Problem: the checklist gives the path, the checkpoint gives verification, but the new hire still does not dare to touch anything - afraid of breaking it. In an organization without rollback, new hires do not dare to change things; if they do not dare to change, they cannot learn; if they cannot learn, the founder still carries everything alone.

Solution: rollback = change is reversible. Three concrete actions:

1. Version control: commit before you change, so you can go back if you break it. Our knowledge base (/root/weiwuji-knowledge-base) makes every change a commit:

git log --oneline -5
# fa0676f deep-read notes from 5 WeChat articles
# 11888df complete deep-analysis report v1.0

Broke something? git revert brings you back to the last stable version in seconds.

2. Publishing can be re-uploaded: a wrong draft is not a disaster - just re-upload it. The real artifact is reorder_drafts.py, which re-uploads drafts in reading order; the ledger (publication-ledger.md) verifies the draft box on every check (draft/batchget), and when a draft is missing from the box it is marked "confirm needed" and waits for human confirmation - never silently overwritten.

3. Errors can be logged: every rollback is a piece of knowledge. error-ledger.md logs in four parts: symptom → root cause → fix → status. A real entry:

## 2026-08-02 · Daily report trend date shows "?" (fallback fix for missing collected_at)
**Symptom:** trend date in the daily report shows a question mark
**Root cause:** collected_at field missing
**Fix:** add date fallback logic - when collected_at is missing, fall back to the entry-level timestamp
**Status:** fixed

✅ Verification: git revert is one command; after logging an error, grep finds the answer in 10 seconds.

🩸 Pitfall: rollback fails when it is "built but never used". We paid for this: the overseas duplicate-publish incident happened precisely because nobody had recorded "Draft not found actually means the publish succeeded", and the retry logic created duplicates. Now every error is logged the same day it happens.

💼 Direct value: rollback compresses the cost of trying from hours to minutes. New hires dare to practice on the real system - and the more they practice, the faster they learn.

▸ Cognitive shift: rollback is essentially turning "I broke it" from an incident into a learnable experiment - experiments are allowed to fail, as long as every failure is recorded.

checkpoint fails -> revert to last stable -> log to error ledger -> fix and retry; every rollback produces one error record" width="800" height="480">

Before vs After: one-week handover vs three months of mentoring

Stage	Before: writing documents	After: the landing trio
Learning path	The new hire digs through the shared drive, guessing	Checklist items checked off one by one, path explicit
Did they learn it	Ask "do you understand?", he says yes	Gate verifies evidence, artifacts speak
Something broke	Wait for the founder to rescue, system stalls	`git revert` / re-upload, recovery in minutes
Trial mindset	Afraid, does not dare to touch	Dares - mistakes can be rolled back and learned from
While you are on vacation	The new hire calls you	Checklist + gates + rollback run by themselves

Why the trio beats "more training"

A lot of people assume SOP rollout equals running a few more training sessions. Wrong direction again.

Training solves "do you know it"; the trio solves "can you, and do you dare". The checklist turns "what to learn" into executable tasks, the checkpoint turns "did you learn it" into verifiable evidence, and rollback turns "trial and error" into a low-cost action - together, they form a learning environment that allows failure.

Go one layer deeper: an organization can give trust to newcomers in two ways. One is betting on character - "I bet you will read it carefully". The other is betting on mechanism - "whether you read it or not, the evidence will speak". Mechanism-based trust does not need a bet, so it scales: a new hire arrives, copy a checklist, run the same gates, enjoy the same rollback net.

This is the second real question of digital transformation: the carrier of organizational capability evolves from "documents" to "mechanisms". In the previous article we turned experience into documents; in this one we turn documents into mechanisms. Documents go stale; mechanisms self-correct - because every rollback flows back through the error ledger into a new rule.

Summary: 3 things you can start today

Write an onboarding checklist: break your system's "what to learn" into 3 days of checkable tasks, each with one acceptance command
Change "did you read it" to "is the evidence there": give your key documents a check script, and block when evidence is missing
Build one rollback action: put the system under git (or equivalent version control) first, then create an error ledger - log today's pitfall today

Do these three, and your documents get "used" for the first time. This is the second foundation stone of digital transformation - first make the documents get used, then talk about upgrading the organization.

Next article: Digital Transformation in Practice, Part 3 - From SOP to Immunity: Turning Every Incident into Organizational Capability

The trio lets a new hire take over in one week, but incidents will still come. Next we upgrade the error ledger from "personal debugging" to "organizational immunity": incident → log → fix → feed back into the rules, so the system grows a memory every time it makes a mistake - this is what Loop Engineering looks like at the organizational level.

🧭 Cognitive Index

🏷️ Entities: onboarding-checklist · gate-check.sh · error-ledger · reorder_drafts.py
💼 Value: SOP rollout · one-week handover · organizational mechanisms
🧠 Cognition: from "writing good documents" to "making documents get used"

"Stop Testing Final Answers: Trajectory Evals Are the Truth About Agent Quality"

weiwuji — Thu, 13 Aug 2026 02:57:01 +0000

The Pain: You test your agent's final answers — but agents misbehave in the trajectory, not the answer. Wrong tool called, wrong params passed, three loops, a guardrail silently bypassed. Answer all green, process all wrong.
What You'll Learn: 2026's biggest shift in evaluation methodology — from testing outputs to testing trajectories (Trajectory Evals) — plus a copy-paste TrajectoryEvaluator you can run today.

A Scenario That Makes Your Back Crawl

Here's a real case (it happened in my system):

User: check this client's quote history

Agent trajectory (every step logged):
① search_contact("Shanghai Logistics") → 3 candidates found
② get_quote_history(contact_id=007) → empty
③ get_quote_history(contact_id=003) → 12 quotes returned
④ send_email(to=007, content=003's quotes) ← wrong here!
⑤ return "sent"

Final answer: ✅ "Quote history sent to client"

Final answer all green. But step ④ sent contact 003's quotes to contact 007 — wrong recipient.

This isn't my invention. 2026's evaluation community has two consensus-level quotes:

Anthropic (2026.01, "Demystifying evals for AI agents"):
"We've seen teams with 90% benchmark scores fail in production."

morphllm:
"A final-answer score can be green while the trajectory looped and three turns drifted off policy."

Core insight: output-level testing is blind to agents. You test what it said; the agent's real work happens in what it did.

Answer green ≠ trajectory correct: an agent's value lives in the process, not the conclusion.

Why Testing Trajectories Beats Testing Answers

Traditional (output-level) testing looks at:

Input → Agent → final answer
                  ↑
            only test here

Trajectory evals look at:

Input → ①call tool A → ②call tool B → ③decide → ④output
          ↑              ↑            ↑
   right tool?     right params?    loops?

Agent "errors" mostly live in the trajectory, not the answer:

Error type	Answer looks	Trajectory shows
Wrong tool	normal	wrong selection ✅caught
Wrong params	normal	bad arguments ✅caught
Infinite loop	timeout/empty	loop count anomaly ✅caught
Privilege escape	normal	out-of-whitelist access ✅caught
Guardrail bypass	normal	skipped check node ✅caught

One line: the answer tells you the result; the trajectory tells you why — and agent reliability problems all live in the why.

My Practice: The error-ledger Data Flywheel

In my system, evaluation isn't a one-time test — it's a continuously spinning data flywheel:

Production agent execution
  ↓
Every step logged to audit trail (action/params/result/time)
  ↓
Problem found → error recorded (error-ledger, 31 entries)
  ↓
Lesson extracted → sedimented as rule/skill (writing_lessons, 50 lines)
  ↓
Gate regression intercepts (publish_gate: same error blocked next time)
  ↓
Back to step 1 (flywheel spins)

This is trajectory evaluation made concrete: not a paper concept, a production system already turning.

error-ledger → lessons → publish_gate: the evaluation flywheel keeps spinning.

Copy-Paste: A TrajectoryEvaluator You Can Run

Here's the trajectory evaluator actually running in my system (simplified, copy-paste ready):

"""TrajectoryEvaluator: log every step, detect anomaly patterns"""
from dataclasses import dataclass, field
from typing import List, Dict, Any
import json
import time

@dataclass
class TrajectoryStep:
    """One step in a trajectory"""
    action: str                 # action: tool/function called
    params: Dict[str, Any]      # params: passed to tool
    result: Dict[str, Any]      # result: returned by tool
    timestamp: float = field(default_factory=time.time)

@dataclass
class Trajectory:
    """One full execution's trajectory"""
    task_id: str
    steps: List[TrajectoryStep] = field(default_factory=list)

    def add_step(self, action, params, result):
        self.steps.append(TrajectoryStep(action, params, result))

    def summary(self):
        return [f"{s.action}({json.dumps(s.params, ensure_ascii=False)[:50]})" 
                for s in self.steps]

class TrajectoryEvaluator:
    """Trajectory evaluator: detect anomaly patterns"""

    # Rule 1: same tool called N+ times consecutively = possible loop
    MAX_REPEAT = 3

    def __init__(self):
        self.whitelist = set()  # tool whitelist (least privilege)
        self.violations = []

    def evaluate(self, traj: Trajectory) -> Dict[str, Any]:
        """Evaluate a trajectory, return {score, violations}"""
        self.violations = []

        # Check 1: loop detection
        actions = [s.action for s in traj.steps]
        for i in range(len(actions) - self.MAX_REPEAT + 1):
            window = actions[i:i + self.MAX_REPEAT]
            if len(set(window)) == 1:  # same tool consecutively
                self._violate(f"possible loop: {window[0]} called {self.MAX_REPEAT}x")

        # Check 2: whitelist detection (privilege escape)
        for s in traj.steps:
            if self.whitelist and s.action not in self.whitelist:
                self._violate(f"privilege escape: called non-whitelisted tool {s.action}")

        # Check 3: param sanity (calling tool with empty required params)
        for s in traj.steps:
            if s.action in ("send_email", "send_quote") and not s.params.get("to"):
                self._violate(f"param error: {s.action} missing recipient")

        # Score: -20 per violation, start 100
        score = max(0, 100 - len(self.violations) * 20)
        return {
            "score": score,
            "passed": score >= 80,
            "violations": self.violations,
            "trajectory": traj.summary(),
        }

    def _violate(self, msg):
        self.violations.append(msg)
        # Feed the error-ledger flywheel
        print(f"  WARNING VIOLATION: {msg}")

# Usage
ev = TrajectoryEvaluator()
ev.whitelist = {"search_contact", "get_quote_history", "send_email"}

# Correct trajectory
good = Trajectory(task_id="T1")
good.add_step("search_contact", {"name": "Shanghai Logistics"}, {"candidates": [{"id": 7}]})
good.add_step("get_quote_history", {"contact_id": 7}, {"quotes": 12})
good.add_step("send_email", {"to": 7}, {"sent": True})

# Wrong trajectory (wrong recipient: step 3 uses 003's data for 007)
bad = Trajectory(task_id="T2")
bad.add_step("search_contact", {"name": "Shanghai Logistics"}, {"candidates": [{"id": 7}]})
bad.add_step("get_quote_history", {"contact_id": 3}, {"quotes": 12})
bad.add_step("send_email", {"to": 7, "content": "003's quotes"}, {"sent": True})

print("=== Correct trajectory ===")
print(json.dumps(ev.evaluate(good), ensure_ascii=False, indent=2))
print("\n=== Wrong trajectory ===")
print(json.dumps(ev.evaluate(bad), ensure_ascii=False, indent=2))

Run it and you'll see the difference: the correct trajectory scores 100; the wrong one — despite an "all-green" answer — gets caught by the param-sanity rule (send_email missing recipient style anomaly). That's trajectory evaluation's value.

Loop / whitelist / param sanity: three rule layers score the trajectory.

Wiring Trajectory Evals Into Your System (3 Steps)

Step 1: Add "trajectory logging" to every tool call

Whatever framework (LangGraph/CrewAI/self-built), the core is every step records action/params/result.

Step 2: Define your anomaly rules

Start with my three, then grow:

Loop (same tool N consecutive times)
Privilege escape (non-whitelisted tool)
Param error (missing required / bad format)
Latency anomaly (single step timeout)

Step 3: Hook into gate + data flywheel

Low score → block output → human confirms
Confirmed error → error-ledger → extract rule → back to step 2

Where You Are Now

You're no longer the optimistic developer who assumes "answer all green means agent is fine."

You're becoming the strict engineer who audits the trajectory, scores every action, and sediments every error type into rules.

2026's biggest evaluation consensus: test the trajectory, not the answer. Because the agent's value lives in the process — answers lie, trajectories don't.

Remember: an all-green final answer doesn't mean the agent didn't do something wrong. Move evaluation from output to trajectory, and you'll finally see what the agent is really doing.

"DeepSeek-V4 Pro: Everyone Is Benchmarking It, I'm Running It in Real Business Scenarios"

weiwuji — Thu, 13 Aug 2026 02:56:48 +0000

The Pain: DeepSeek-V4 launched and much of the conversation centers on "near Fable5" and "price is so low" — and that's fair, benchmarks and price are essential selection data. But I want to share what it means for the agent systems you're building in real business scenarios.
What You'll Learn: The 2026 industry shift — from raw benchmark performance to landed value — plus the real signal seen by someone actually running it in real business scenarios.

Let's Align on Facts: What V4 Actually Launched

On August 13, 2026, DeepSeek-V4-Pro-0813 officially landed (the API version number IS today's date; Hugging Face concurrently published the 1.6T-parameter model). This isn't a roadmap announcement — it just shipped.

I checked the official API docs. Hard facts, no hype:

Spec	deepseek-v4-flash	deepseek-v4-pro
Version	V4-Flash-0731	V4-Pro-0813 (released today)
Parameters	291B	1.6T
Context	1M tokens	1M tokens
Max output	384K	384K
Tool calls	✅	✅
JSON output	✅	✅
Anthropic API compat	✅	✅
Input (cache hit)	$0.0028/1M	$0.0036/1M
Input (cache miss)	$0.14/1M	$0.435/1M
Output	$0.28/1M	$0.87/1M

Benchmark picture (official release framing): DeepSeek-V4-Pro scores near Fable5, close to SOTA, in the same tier as Grok's top models — especially on million-token long-context comprehension and reasoning. The paper is literally titled "DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence."

Pricing in CNY (official): V4 Pro cache-miss input is 3 CNY/1M tokens, output 6 CNY/1M (3x Flash); concurrency limit 500 (Flash 2500). DeepSeek's own positioning is literally "built for Coding and complex Agent tasks" — note the emphasis isn't benchmarks, it's agent tasks.

A key detail most people will miss: the official announcement says prices will rise significantly soon. The current price is a window of opportunity.

Input price ~1/100 of GPT/Claude class (log scale).

1M context + tool calls + $0.0028/1M cache-hit input: agent compute costs just collapsed.

Everyone Is Benchmarking. I Want to Talk About Production Use.

Search for DeepSeek-V4 and you'll find many benchmark comparisons and price analyses — and that's valuable, benchmarks and price are the foundation of any model selection.

The question I want to explore: can it survive a real business running 24×7?

But notice the trend shift: in 2026, the entire AI industry is moving from "raw benchmark performance" to "landed value." Even DeepSeek's own positioning isn't "how high the benchmark" — it's literally "built for Coding and complex Agent tasks." The vendor itself isn't selling benchmarks anymore; it's selling agent-task capability.

Benchmarks measure the ceiling; production measures the floor. A model's real value is set by its floor.

My Practice: Real Business Scenarios Run on V4

The real business scenarios I work on now run entirely on DeepSeek:

Zhi Mo (my AI operations partner) → deepseek-v4-flash
Logistics agent (rates/quotes/reports) → deepseek-v4-flash
Content pipeline (writing/translation/image captions) → flash + pro
Multi-platform publishing → orchestrated by Zhi Mo

Three Real Experiences

1. 1M context isn't a gimmick — it's genuinely less work
Before: split long docs into chunks, summarize, build RAG. Now: stuff the whole document in, agent reads everything at once, then works. I deleted three "chunking" middle steps.

# Before: manually chunk the document
chunks = split_document(doc, max_chunk=8000)
for c in chunks:
    summary += summarize(c)

# After: stuff the whole thing in
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[{"role": "user", "content": full_doc}],  # whole document
    thinking={"type": "enabled"},
)

2. Tool calls are the foundation of agent engineering
V4's tool calls + Anthropic API compatibility mean: Claude Code, OpenCode, GitHub Copilot — change one line of base_url and they use DeepSeek as backend. No code rewrite.

3. Cost: from "counting tokens" to "just use it"
Cache-hit input at $0.0028/1M — roughly 1/100 of GPT-class pricing. Before, I weighed token costs for every agent task. Now I let agents try more rounds, think more steps.

One line of base_url swaps your entire agent toolchain to DeepSeek backend.

The Differentiating Insight: The Model Is the Engine, Engineering Is the Car

Here's the core of what I want to say.

Benchmarks measure engine horsepower. But whether a car drives well depends on the chassis, gearbox, brakes — that's engineering.

Same deepseek-v4-flash:

Raw use (pure prompt) → outputs are hit-or-miss
Engineered use (tool isolation + scene routing + observability + correction sedimentation) → reliable, 24×7

Our practice (covered in previous articles):

Scene routing: don't make one agent do everything
Tool whitelist: minimal permissions per scene
Correction sedimentation: never make the same mistake twice
Supervisor orchestration: orderly multi-agent collaboration

The cheaper the model, the more valuable engineering becomes. Cheap models let you run more agents and more rounds — but without engineering, more runs means more mistakes.

Benchmarks decide entry; real value decides whether it's worth using.

These practices from real business scenarios apply directly to OPC (one-person companies) and formal company operations — whether you're running a business solo or driving digital transformation inside an organization, this engineering toolkit (scene routing + tool whitelist + observability + correction sedimentation) transfers as-is. DeepSeek-V4 collapsing compute costs is exactly the moment to scale it.

Model prices drop → engineering value rises. Cheap models enable more agents and more rounds; only engineering makes them stable.

Your Action Items

Seize the window: the official notice says prices will rise significantly. Now is the best time to deploy or migrate
Try small first: switch one non-critical agent to v4-flash for two weeks; judge by real data, not benchmarks
Don't use it raw: even the strongest model needs scene routing + tool whitelist + observability (our series C and D)
Run the math: calculate your actual scenario at cache-hit prices — it's probably an order of magnitude cheaper than your current setup

Where You Are Now

You shouldn't be the spectator who "saw the DeepSeek-V4 news, read the benchmarks, closed the tab."

You should be the practitioner who "sees the opportunity, wires the model into their own system, and validates it with real business."

The real news about DeepSeek-V4 isn't the benchmark score — it's an engine that cuts agent production costs by 90%, currently on sale, and about to get more expensive.

The engine has arrived. Is your car built?

Stop Testing Final Answers: Trajectory Evals (Merged)

weiwuji — Thu, 13 Aug 2026 01:29:16 +0000

⚠️ This article has been merged into the updated version: Stop Testing Final Answers: Trajectory Evals Are the Truth About Agent Quality

The Pain: You test your agent's final answers — but agents misbehave in the trajectory, not the answer. Wrong tool called, wrong params passed, three loops, a guardrail silently bypassed. Answer all green, process all wrong.
What You'll Learn: 2026's biggest shift in evaluation methodology — from testing outputs to testing trajectories (Trajectory Evals) — plus a copy-paste TrajectoryEvaluator you can run today.

A Scenario That Makes Your Back Crawl

Here's a real case (it happened in my system):

User: check this client's quote history

Agent trajectory (every step logged):
① search_contact("Shanghai Logistics") → 3 candidates found
② get_quote_history(contact_id=007) → empty
③ get_quote_history(contact_id=003) → 12 quotes returned
④ send_email(to=007, content=003's quotes) ← wrong here!
⑤ return "sent"

Final answer: ✅ "Quote history sent to client"

Final answer all green. But step ④ sent contact 003's quotes to contact 007 — wrong recipient.

This isn't my invention. 2026's evaluation community has two consensus-level quotes:

Anthropic (2026.01, "Demystifying evals for AI agents"):
"We've seen teams with 90% benchmark scores fail in production."

morphllm:
"A final-answer score can be green while the trajectory looped and three turns drifted off policy."

Core insight: output-level testing is blind to agents. You test what it said; the agent's real work happens in what it did.

Answer green ≠ trajectory correct: an agent's value lives in the process, not the conclusion.

Why Testing Trajectories Beats Testing Answers

Traditional (output-level) testing looks at:

Input → Agent → final answer
                  ↑
            only test here

Trajectory evals look at:

Input → ①call tool A → ②call tool B → ③decide → ④output
          ↑              ↑            ↑
   right tool?     right params?    loops?

Agent "errors" mostly live in the trajectory, not the answer:

Error type	Answer looks	Trajectory shows
Wrong tool	normal	wrong selection ✅caught
Wrong params	normal	bad arguments ✅caught
Infinite loop	timeout/empty	loop count anomaly ✅caught
Privilege escape	normal	out-of-whitelist access ✅caught
Guardrail bypass	normal	skipped check node ✅caught

One line: the answer tells you the result; the trajectory tells you why — and agent reliability problems all live in the why.

My Practice: The error-ledger Data Flywheel

In my system, evaluation isn't a one-time test — it's a continuously spinning data flywheel:

Production agent execution
  ↓
Every step logged to audit trail (action/params/result/time)
  ↓
Problem found → error recorded (error-ledger, 31 entries)
  ↓
Lesson extracted → sedimented as rule/skill (writing_lessons, 50 lines)
  ↓
Gate regression intercepts (publish_gate: same error blocked next time)
  ↓
Back to step 1 (flywheel spins)

This is trajectory evaluation made concrete: not a paper concept, a production system already turning.

error-ledger → lessons → publish_gate: the evaluation flywheel keeps spinning.

Copy-Paste: A TrajectoryEvaluator You Can Run

Here's the trajectory evaluator actually running in my system (simplified, copy-paste ready):

"""TrajectoryEvaluator: log every step, detect anomaly patterns"""
from dataclasses import dataclass, field
from typing import List, Dict, Any
import json
import time

@dataclass
class TrajectoryStep:
    """One step in a trajectory"""
    action: str                 # action: tool/function called
    params: Dict[str, Any]      # params: passed to tool
    result: Dict[str, Any]      # result: returned by tool
    timestamp: float = field(default_factory=time.time)

@dataclass
class Trajectory:
    """One full execution's trajectory"""
    task_id: str
    steps: List[TrajectoryStep] = field(default_factory=list)

    def add_step(self, action, params, result):
        self.steps.append(TrajectoryStep(action, params, result))

    def summary(self):
        return [f"{s.action}({json.dumps(s.params, ensure_ascii=False)[:50]})" 
                for s in self.steps]

class TrajectoryEvaluator:
    """Trajectory evaluator: detect anomaly patterns"""

    # Rule 1: same tool called N+ times consecutively = possible loop
    MAX_REPEAT = 3

    def __init__(self):
        self.whitelist = set()  # tool whitelist (least privilege)
        self.violations = []

    def evaluate(self, traj: Trajectory) -> Dict[str, Any]:
        """Evaluate a trajectory, return {score, violations}"""
        self.violations = []

        # Check 1: loop detection
        actions = [s.action for s in traj.steps]
        for i in range(len(actions) - self.MAX_REPEAT + 1):
            window = actions[i:i + self.MAX_REPEAT]
            if len(set(window)) == 1:  # same tool consecutively
                self._violate(f"possible loop: {window[0]} called {self.MAX_REPEAT}x")

        # Check 2: whitelist detection (privilege escape)
        for s in traj.steps:
            if self.whitelist and s.action not in self.whitelist:
                self._violate(f"privilege escape: called non-whitelisted tool {s.action}")

        # Check 3: param sanity (calling tool with empty required params)
        for s in traj.steps:
            if s.action in ("send_email", "send_quote") and not s.params.get("to"):
                self._violate(f"param error: {s.action} missing recipient")

        # Score: -20 per violation, start 100
        score = max(0, 100 - len(self.violations) * 20)
        return {
            "score": score,
            "passed": score >= 80,
            "violations": self.violations,
            "trajectory": traj.summary(),
        }

    def _violate(self, msg):
        self.violations.append(msg)
        # Feed the error-ledger flywheel
        print(f"  WARNING VIOLATION: {msg}")

# Usage
ev = TrajectoryEvaluator()
ev.whitelist = {"search_contact", "get_quote_history", "send_email"}

# Correct trajectory
good = Trajectory(task_id="T1")
good.add_step("search_contact", {"name": "Shanghai Logistics"}, {"candidates": [{"id": 7}]})
good.add_step("get_quote_history", {"contact_id": 7}, {"quotes": 12})
good.add_step("send_email", {"to": 7}, {"sent": True})

# Wrong trajectory (wrong recipient: step 3 uses 003's data for 007)
bad = Trajectory(task_id="T2")
bad.add_step("search_contact", {"name": "Shanghai Logistics"}, {"candidates": [{"id": 7}]})
bad.add_step("get_quote_history", {"contact_id": 3}, {"quotes": 12})
bad.add_step("send_email", {"to": 7, "content": "003's quotes"}, {"sent": True})

print("=== Correct trajectory ===")
print(json.dumps(ev.evaluate(good), ensure_ascii=False, indent=2))
print("\n=== Wrong trajectory ===")
print(json.dumps(ev.evaluate(bad), ensure_ascii=False, indent=2))

Loop / whitelist / param sanity: three rule layers score the trajectory.

Wiring Trajectory Evals Into Your System (3 Steps)

Step 1: Add "trajectory logging" to every tool call

Whatever framework (LangGraph/CrewAI/self-built), the core is every step records action/params/result.

Step 2: Define your anomaly rules

Start with my three, then grow:

Loop (same tool N consecutive times)
Privilege escape (non-whitelisted tool)
Param error (missing required / bad format)
Latency anomaly (single step timeout)

Step 3: Hook into gate + data flywheel

Low score → block output → human confirms
Confirmed error → error-ledger → extract rule → back to step 2

Where You Are Now

You're no longer the optimistic developer who assumes "answer all green means agent is fine."

You're becoming the strict engineer who audits the trajectory, scores every action, and sediments every error type into rules.

2026's biggest evaluation consensus: test the trajectory, not the answer. Because the agent's value lives in the process — answers lie, trajectories don't.

Remember: an all-green final answer doesn't mean the agent didn't do something wrong. Move evaluation from output to trajectory, and you'll finally see what the agent is really doing.