Most agent systems answer one question: how do I get the model to write something for me? That is the wrong first question. The right one is: what kind of cognitive work should the agent do at all?
After building and running a research system that has processed 7,243 documents into 138,000 knowledge nodes, 588,000 links, and 966 reports, I can tell you the framework that survived contact with production. It has two parts: four cognitive operations and a flywheel.
The four cognitive operations
When I ask an agent to help with knowledge work, I want it to do exactly one of four things:
- Systematize — take scattered material and organize it into a coherent structure. Not create new claims; just make what already exists navigable and comparable.
- Theorize — connect specific observations to general principles. This is the hardest one and the one where agents hallucinate most, because it requires restraint: only link what the evidence supports.
- Academicize — bring scholarly rigor to raw arguments: citations, methods, evidence chains, counter-arguments. The agent's job is not to be smart; it is to be checkable.
- Systematize (again, but as architecture) — take working pieces and turn them into a reproducible pipeline. This is where "agent writes code" becomes "agent maintains a system." Every feature request gets tested against these four. If a proposed capability does not clearly strengthen one of them, it does not get built. This single filter has killed more features than any budget constraint.
The flywheel
The four operations describe what agents do. The flywheel describes how knowledge compounds:
Documents → Extract → Research → Discover → Challenge → Feed back → New Research
Each step feeds the next:
- Extract: turn raw documents into structured knowledge nodes with verifiable source links.
- Research: use those nodes to generate new analyses on topics a human chooses.
- Discover: identify patterns, collisions, and gaps the research pipeline surfaces automatically.
- Challenge: have a separate pipeline question those discoveries — not just "is this correct?" but "is this new?"
- Feed back: unresolved challenges become new research inputs. The wheel turns. The flywheel is not a metaphor. Each turn measurably increases the knowledge base: more nodes, more links, more reports, more challenges. After several months of continuous operation, the numbers compound.
The gate question
Before any batch of work starts, the agent must answer four questions — and if the answers do not align with the design, the batch does not run:
- Which cognitive operation does this strengthen?
- Which flywheel stage does it feed?
- Which evolution line does it advance — knowledge breadth or method depth?
- What are the three expected outcomes: more truthful, less human effort, more consumption? This is not bureaucracy. It is the cheapest form of strategic alignment I have found: four lines, written before the work starts, checked against the master design. It prevents the most common failure mode of agent systems — drifting into producing impressive-looking output that nobody asked for.
Why "agent as editor, not author" matters
The deepest architectural lesson came from a failure. Early versions of our research pipeline let the writing agent see the full source text. It would then paraphrase everything — technically correct, but the output was just the source material with different words.
The fix was to change the agent's role: editor, not author. The agent receives curated material nodes and is told: "The material below is the only evidence source for this section. Understand it, digest the core arguments, and organize them into a coherent narrative in your own reasoning language." The agent organizes; the material provides the substance.
This distinction — editor vs. author — is the difference between a system that generates text and a system that generates knowledge.
The measurement connection
Here is where this connects to my previous posts on trust and validity states. The flywheel only compounds if each turn produces output you can trust. And trust is not a feeling — it is measurable:
- Are citations reconciled? (citation reconciliation rate)
- Are method claims backed by data? (method honesty rate)
- Are quality scores computed on the final artifact, not a prefix? (measurement-window validity)
- Are absent events reported as unknown, not zero? (the absent-zero rule) The flywheel tells you what to produce. The validity rules tell you whether you can believe it. A system without both is either productive but untrustworthy, or trustworthy but sterile.
What to take
If you are building an agent system for knowledge work, take three things:
- Classify cognitive operations before writing prompts. "Write me an article" is not a cognitive operation. "Systematize these 50 sources into a framework" is.
- Make the agent an editor, not an author. Give it material, not a blank page. The output will be less impressive and more useful.
- Put a gate question before every batch. Four lines about which operation, which flywheel stage, which evolution line, what expected outcome. If the answers are fuzzy, the work will be fuzzy too. The full system that these principles came from processes tens of thousands of documents on real research topics. It is not a demo. The cognitive operations and the flywheel are what survived.
Top comments (0)