DEV Community: joinwell52

Six AI Teams, One Inspection Task: Delivery, Failure, and the Evidence Between

joinwell52 — Thu, 10 Sep 2026 05:24:12 +0000

For developers building multi-agent workflows, a final answer is only one part of delivery. This is the complete field report from six CodeFlowMu/FCoP inspection runs, following task graphs, tool receipts, report submissions, authorized recovery, and an independent evaluator.

The same inspection task. One AI team delivered in 12 minutes 20 seconds. Another was still unfinished after two hours. One never assigned a single child task.

Was the difference model capability, integration, or how the PM organized the work? We gave six AI teams the same assignment through CodeFlowMu and FCoP, then followed the retained task, execution, report, authorization, and independent EVAL records to find out.

Using CodeFlowMu and FCoP to make AI teamwork observable, documented, and open to scrutiny

Runs: September 9, 2026 · Analysis: September 10, 2026

On September 9, 2026, we used CodeFlowMu to give six AI teams the same system-inspection task. Each PM had to divide the work, use tools, collect specialist reports, and report to ADMIN. The central question was not simply which team finished fastest. It was what evidence CodeFlowMu captured, and how that evidence let us compare, diagnose, and judge the work. FCoP provided formal coordination records; CodeFlowMu connected tasks, execution, delivery, approvals, recovery, and independent EVAL analysis. Both successful and failed attempts remained available for review.

1. CodeFlowMu, FCoP, and the Test

1.1 What CodeFlowMu and FCoP do

CodeFlowMu is a multi-agent team coordination and governance system built on FCoP, with locally retained work records. It organizes agents in different jobs and supports formal delivery, evidence checking, approval, independent evaluation, runtime diagnosis, and evidence retention. Users can trace a task to its execution, then trace a report's claims back to supporting records.

FCoP, the File-based Coordination Protocol, expresses collaboration through formal files. TASK, REPORT, and related review records represent assignments, deliverables, and decisions in a persistent, inspectable form. CodeFlowMu applies that protocol to operating AI teams. See the public FCoP project.

Together, they address a practical problem: after someone asks an AI team to “check the system,” who actually checked it, what was examined, and what supports the claim that the work is complete? Task relationships, execution receipts, and reports make these questions answerable after the conversation has ended.

Who executes, and who evaluates?

The test configuration had one human ADMIN, four execution-team seats, and an independent EVAL seat. The table describes job responsibilities. Each PM decided how to divide the five inspection areas between DEV, OPS, and QA; those differences were part of the test.

Role	Responsibility	Main evidence
ADMIN	Human requester and final acceptor; assigns work, authorizes restricted actions, and can terminate a run	Root submission, authorization, acceptance, and archive records
PM-01	Interprets the request, creates assignments, coordinates progress, accepts child work, and reports to ADMIN	Child TASKs, dispatch and acceptance receipts, final PM REPORT
DEV-01	Performs assigned technical, tool, or code-related inspections	Tool/execution records and DEV REPORT; inspection is not automatic authorization to modify code
OPS-01	Checks assigned runtime, environment, and configuration concerns	Runtime evidence and OPS REPORT
QA-01	Performs assigned verification and review, distinguishing supported conclusions from issues and unknowns	QA REPORT and its actual verdict
EVAL-01	Independently examines the run, delivery quality, and evidence consistency	Observation report, run-record analysis, and evaluation-attempt records

All six runs used the same EVAL configuration: Cursor SDK / auto-smart. EVAL did not switch with the tested team. PM, DEV, OPS, and QA were the execution team; EVAL ran in its own session, did not perform the team's inspection assignments, and did not make PM or ADMIN acceptance decisions. Holding evaluator configuration constant reduced one source of variation. auto-smart is a routing label, however, not proof that the underlying foundation model stayed fixed.

EVAL has three distinct report paths: a task-run record analyzes a complete execution, a system observation examines system assets, and a closeout observation checks PM’s final report against its task evidence chain. All contain analysis, with different triggers, scopes and skill combinations; see the report and skill matrix in section 1.10. We check EVAL against original records and retain failures, retries and mismatched material. A common evaluator configuration does not mean that every run produced all three reports.

English reading guide — original Chinese UI: “待 ADMIN 验收” means “Awaiting ADMIN acceptance”; “子任务已完成” means “Child task completed”; “未投递 REPORT” means “Undelivered REPORT.” The screenshot preserves the conflict between completed tasks and stale queue labels.

Click the screenshot to inspect its original pixels.

Original screenshot, September 9: the root task awaits ADMIN acceptance while three specialist tasks are complete. The stale “undelivered report” message at the bottom was separately recorded as a display issue. A UI label cannot replace formal delivery receipts. Screenshots preserve the original Chinese interface; the surrounding English text explains the relevant evidence.

1.2 The exact task

All six runs received the same initial task body. Reminders, authorizations, and manual termination were recorded separately; this was not presented as an intervention-free experiment.

Original ADMIN task, preserved verbatim:

检查FCoP落地情况，检查MCP工具情况，检查SKILLS分配和使用，检查各角色权限和职责，检查轨机运行情况；请PM分解任务，团队协作完成；最后形成报告向ADMIN汇报！

English translation:

Check the implementation of FCoP, the MCP tools, the allocation and use of SKILLS, each role's permissions and responsibilities, and runtime operation. PM should decompose the task and have the team complete it collaboratively, then produce a report for ADMIN.

Task titles identified the tested model—for example, “system inspection codex,” “system inspection Doubao,” or “system inspection Qwen.” The shared test was the body above. The original wording “轨机” is retained in the source rather than silently edited.

Why this task?

First, it inspects the environment the team itself uses: coordination records, available tools, skill allocation, permissions, and runtime health. Second, it explicitly requires decomposition, teamwork, and a final report, exposing organization as well as individual reasoning. Third, it leaves realistic natural-language choices open. We did not preassign every checklist item or dependency, allowing us to observe how each PM chose scope, sequence, and a stopping point.

A reasonable deliverable is an evidence-backed inspection report. Each of the five areas needs a scope, a verification method, findings, and any unverified items. Workers should provide formal reports and the PM should form an overall judgment. Finding a system issue does not automatically mean the inspection failed. Conversely, “inspect and report” does not automatically authorize code repair. Inspection completion and product health are separate outcomes.

1.3 CodeFlowMu as the test instrument

CodeFlowMu organized the work and retained its evidence. It did not replace the PM's business judgment.

The workflow was ADMIN → PM → DEV/OPS/QA → PM delivery, with independent EVAL examining the records. The instrument had three roles: operate the collaboration, collect evidence of what happened, and support independent review. All timing, completion, and quality comparisons follow that record chain.

1.4 Models and integration paths

“Six AIs” refers to six models operating as teams under their respective integration configurations. They ran in this order: Codex, Doubao, DeepSeek, Kimi, Qwen, Cursor. Model identifiers came from the identity replies and saved run records.

Scheme	Model identifier	Provider and execution framework
Codex	`gpt-5.6-terra`	ChatGPT subscription / Codex CLI (H2 · Shadow)
Doubao	`doubao-seed-2-0-pro-260215`	Ark API / Codex app-server
DeepSeek	`deepseek-v4-pro`	DeepSeek API / Codex app-server
Kimi	`kimi-k3`	Moonshot API / Codex app-server
Qwen	`qwen3.8-max`	DashScope API / Codex app-server
Cursor	`auto-smart`	Cursor SDK

The first five used the Codex framework; Cursor used its SDK. The Cursor QA seat initially had a model unavailable through that Host, and recovered after authorization. That intervention remains part of the result. The EVAL configuration stayed Cursor SDK / auto-smart throughout.

English reading guide — original Chinese UI: the top section is “EVAL channel and model.” It shows the PM team using codex/gpt-5.6-terra, EVAL using cursor/auto-smart, and no active EVAL session. The bottom cards are PM (project manager), DEV (developer), OPS (operations), QA (quality assurance), and EVAL (independent evaluator).

Click the screenshot to inspect its original pixels.

This later-supplied configuration screenshot illustrates the product, rather than proving every historical run's configuration. It shows a Codex/gpt-5.6-terra PM team and a separately configured Cursor/auto-smart EVAL seat. No EVAL session is running in the screenshot. Historical model identity must still be checked against the corresponding run.

1.5 Initialization, execution, and timing

Every run began after CodeFlowMu system initialization. Records were exported and backed up before initialization for the next run. We configured and checked the next team's model, restarted the service on port 18766 after switching models, and submitted the same task. Late EVAL reports were preserved in supplementary backup batches.

Condition	Procedure
Machine and code	Same machine, CodeFlowMu V2.2.9, commit `cb590ce`
Starting point	System initialization before each new run
Execution team	Switch and verify the tested configuration; record mistakes and recovery
Evaluation	Keep EVAL at Cursor SDK / auto-smart
Task	Same original body in section 1.2
Scheduled reminder	Leave the initialization-default PM progress reminder enabled; it wakes the PM but makes no business decision
Retention	Separate export and backup by run; correlate sessions as well as possibly reused task IDs

Initialization is not a full snapshot restore of the OS, every cache, or external model services. Network and provider conditions can change. The result therefore compares integrated teams following a shared initialization procedure, not isolated foundation models under perfectly identical conditions.

Duration runs from the PM's first formal session to successful submission of its normal final report. Authorization and recovery within the run count toward elapsed time. Subsequent ADMIN acceptance and EVAL generation do not. For incomplete runs, we report time to forced termination, not “completion time.”

1.6 From a conversation to inspectable work

CodeFlowMu placed assignments, execution, delivery, and authorization into a traceable workflow. The PM's organizational choices became observable: how quickly it delegated, which relationships it created, whether reports were complete, and how it handled trouble. Similar final answers need not mean equally reliable work.

The Cursor run retained failed QA launches, ADMIN authorization, cancellation of the old task, the linked rerun, and final delivery. Qwen retained a different sequence: server-side task creation applied despite transport errors, and a complete OPS report body that failed to become a formal report. This distinguishes saying something is done, attempting to deliver it, and actually delivering it.

The same records also revealed instrument defects: stale projections, report-generation problems, and incorrect raw-material associations. That ability to investigate the instrument itself is valuable; it does not mean every projection is already correct.

1.7 “Files are the truth” means work must be written down

Our analysis uses a retained experimental archive, not an agent's memory or retrospective account. Original CodeFlowMu/Host execution records, FCoP artifacts, independent EVAL reports, and the backups, timelines, checks, and score tables built from them form the evidence base.

Layer	Retained material	Purpose
Original run	TASK, REPORT, sessions, tool returns, public progress text, approvals, issues, logs	Preserve assignments, actions, claims, and authorization as observed
Independent evaluation	Both EVAL report types, failed generations, late supplements	Supply judgments and gaps that can themselves be checked
Preservation	Separate ZIPs, inventories, sizes, SHA256 hashes, source index	Preserve versions and avoid mixing runs
Analysis	Stage timeline, timing tables, fact checks, diagnosis, scores, detailed report	Turn retained observations into reviewable conclusions

We did not ask agents to remember what they had done and then score the recollection. We reconstructed the run, checked task/report links, and compared statements with execution. These files remain useful after sessions end or models change.

The principle is durable documentation: assignments, actions, approvals, delivery, and analysis must be saved, transferable, and traceable. Files can still contain mistakes. A hash fixes the saved bytes; it does not establish the truth of a business claim.

Each run was exported as a separate raw-evidence.zip, with an inventory, sizes, hashes, and ZIP integrity checks. Late Qwen and Cursor EVAL outputs have separate supplements. Runtime evidence comes from CodeFlowMu and its Hosts; independent backup and later review are preservation and analytical steps applied to that evidence.

When records conflict, first align model, time window, Session, task, and report. Initialization can reuse TASK-001 or CUSTOM-001. A shared ID alone does not identify a run. Report text does not prove submission; a “running” display cannot overturn a cancellation receipt; a “pass” claim cannot substitute for the corresponding test.

1.8 The business support around agent work

CodeFlowMu supports what happens after an agent starts working: evidence checking, acceptance by responsible roles, independent observation, and diagnosis. These are connected by actual files. Execution status, verification, PM judgment, and EVAL analysis should not collapse into one vague “the system says complete.”

First, formal submissions and child tasks identify who is responsible. Execution records connect tasks, roles, sessions, calls, returns, and timestamps. Reports have formal writes and delivery records, separating execution, generated text, submission, and acceptance.

Second, REVIEW-GATE creates fact-check records. In the Cursor archive, a review of DEV task 002 includes execution_evidence_state: verified, review_state: needs_pm, business_decision: false, and attention_owner: PM. The evidence had been checked, but business judgment remained with the PM. third_party_source_state: not_configured does not pretend an external source was consulted. Compatibility fields must be read alongside the actual decision owner.

Third, PM accepts child work and ADMIN accepts the root task. Commands have receipts and state transitions; restricted actions have authorization records. EVAL and programmatic checks do not replace that responsibility. Cursor's recovery used this authorization path.

Fourth, EVAL records its observations and attempts. Diagnosis follows task, session, tool, delivery, and approval evidence to determine where a failure occurred. Confirmed issues can become ISSUE records; incomplete evidence remains a hypothesis rather than automatically becoming a product defect.

Support stage	Actual file or file family in the Cursor archive	What can be checked
Submission and assignments	`SUBMISSION-20260909-001.json`, root/child TASKs, `fcop/ledger/tasks.jsonl`	Request, division of labor, cancellation and rerun relationships
Execution	`actions-20260909.jsonl`, `runtime-events-20260909.jsonl`	Actions, sessions, results, and time
Tool transport	`.codeflowmu/logs/tool-transport-events.jsonl`	Call identity and transport events
Delivery	Four formal REPORTs, `.codeflowmu/report-delivery/acks.jsonl`	Content and delivery to a PM session
Fact checking	`REVIEW-…-REVIEW-GATE-on-TASK-….md`	Evidence state, pending decisions, snapshots, responsibility
Decisions and authorization	Task-command receipts, approval audit, GOV record	Requests, applied operations, authorization scope
EVAL	`OBSERVATION-20260909-002-panel-scan.md`, `…003-benchmark-CUSTOM-20260909-001.md`	Independent asset and run analysis
EVAL recovery	`eval-observation-attempts.jsonl`	Starts, failures, retries, and outcomes
Issue tracking	`ISSUE-20260909-001-PM.md` and closure records	How the configuration issue was raised and handled
Process and usage	Chat/task JSONL, usage JSONL	Public progress, Host results, and usage context

The public evidence inventory identifies 26 verified files by name, size, and hash. The complete private logs and credentials are not distributed with this article.

1.9 What real artifacts look like

These are excerpts from the final Cursor archive, not invented templates. YAML fields and report sections are selected for explanation. Omitted fields may contain additional state and findings.

task_id: TASK-20260909-005
root_task_id: TASK-20260909-001
sender: PM
recipient: QA
parent: TASK-20260909-001
depends_on: []
acceptor: PM
rerun_of: TASK-20260909-003
subject: 只读核查 FCoP 落地与角色权限职责边界（模型修复后重跑）

recipient assigns QA, parent and root_task_id connect the root task, and acceptor assigns PM acceptance. rerun_of links new task 005 to old task 003. An empty depends_on indicates no explicit execution dependency. The excerpt establishes identity and relationships, not a passing inspection by itself.

kind: fact_check
task_id: TASK-20260909-002
report_id: REPORT-20260909-001-DEV-to-PM
review_state: needs_pm
execution_evidence_state: verified
business_decision: false
attention_owner: PM
third_party_source_state: not_configured

Verified execution evidence and a pending PM judgment coexist. The review explicitly says it made no business decision. This is how a saved fact check can support acceptance without silently replacing the accepting role.

## 子任务回执
- `REPORT-20260909-001-DEV-to-PM.md` ← `TASK-20260909-002`（approved）
- `REPORT-20260909-002-OPS-to-PM.md` ← `TASK-20260909-004`（approved）
- `REPORT-20260909-003-QA-to-PM.md` ← `TASK-20260909-005`（approved；`rerun_of` 作废的 `TASK-20260909-003`）

## 说明
全程保持只读巡检目标；PM 未修改业务代码/配置。根任务业务验收与归档由 ADMIN 决定。

The PM lists three worker reports and preserves the QA rerun and ADMIN acceptance boundary. The original report text is Chinese and is intentionally retained as evidence: it says DEV, OPS, and the new QA task were accepted, and final root acceptance and archiving belong to ADMIN. These are claims to check against receipts, approvals, and execution—not proof merely because the report says “approved.” See the excerpt source index.

1.10 EVAL: another agent observes, with three distinct report paths

When the execution team says “done,” who checks that claim? In CodeFlowMu, another EVAL agent reads the evidence. PM organizes delivery; DEV, OPS, and QA perform the work. EVAL uses a separate session to inspect their files and execution records, compare claims with evidence, and produce an independent analysis.

All six runs used Cursor SDK / auto-smart for evaluation. Independence means a separate role, session, and responsibility. Even when the tested team also used Cursor, EVAL was another session. This does not establish a different underlying foundation model or guarantee an unbiased, correct judgment. A common configuration helps comparison; evidence checking remains necessary.

Recording starts with the task; model analysis runs when a report is generated

When ADMIN enables recording at task creation, CodeFlowMu creates a run identity and retains the root task, environment and model baseline, start time, and starting log cursors. Existing chat, public progress, tool, session, and business logs retain the execution trail. The EVAL model is not necessarily running continuously or analyzing every event in real time.

Generating the task record freezes the end boundary, recursively identifies the complete child-task tree, gathers the run's material, and starts independent EVAL analysis. This separates continuous evidence retention from model-analysis calls. A run without recording enabled cannot later be presented as a complete prospective record. Existing raw records also do not prove that a final analysis report was generated successfully.

The report matrix: execution history, system assets, and delivery claims

Report	Trigger and scope	What its analysis answers
Task-run record	Recording enabled at creation; generation freezes one root task and its complete tree	How PM assigned work, what happened, timing, retries, interventions and outcomes; evidence and handling of linked ISSUEs
System observation	The EVAL observation action and related observation triggers; a system-asset snapshot	Asset consistency, risks, evidence gaps, emergence patterns, and candidates for knowledge retention
Closeout observation	The root task's EVAL action; PM's final report and its evidence chain	Which delivery claims are supported, contradicted, or unverified, and what ADMIN should inspect before acceptance

Report	Typical file marker
Task-run record	`*-benchmark-<RUN-ID>.md`
System observation	`*-panel-scan.md`
Closeout observation	`*-eval-agent.md`

All three contain analysis. They examine different objects: how a run unfolded, what the system assets show, and whether a delivery report's claims have support. They are not renamed copies of one summary or three automatic approval gates.

System observation covers nine asset classes: ledger, runtime logs, public thinking/progress logs, usage, analytics, internal EVAL, emergence log, role views, and shared knowledge. Public progress means content actually emitted and retained; it does not expose or justify speculation about hidden reasoning.

Which skills support the reports?

Skills are work instructions for the EVAL agent, not additional models. One skill does not correspond to one report. The routing in test commit cb590ce combines seven skills. A checkmark means required injection for that path, not proof that a particular run performed the analysis correctly.

Skill and purpose	Task record	System observation
`eval-statistical-analysis`: freeze identity; calculate timing, calls, failures and retries with formulas, denominators and provenance	✓	—
`eval-issue-analysis`: examine linked ISSUEs, evidence status, impact, and confidence in causal hypotheses	✓	—
`eval-observation-writing`: separate facts, inference, gaps and advice; cite sources and follow the output contract	✓	✓
`eval-admin-closeout-observer`: reconcile PM's final report, worker reports and the current authoritative evidence chain	—	✓
`controlled-emergence-observer`: inspect task relationships for probes, self-tasks, sandboxes and project-tree patterns	—	✓
`eval-risk-gap-analysis`: compare expected and actual behavior and recommend risk severity and ownership	—	✓
`eval-promotion-advice`: recommend findings for follow-up tasks, issue drafts or reusable knowledge	—	✓

Skill and purpose	Closeout
`eval-statistical-analysis`: freeze identity; calculate timing, calls, failures and retries with formulas, denominators and provenance	—
`eval-issue-analysis`: examine linked ISSUEs, evidence status, impact, and confidence in causal hypotheses	—
`eval-observation-writing`: separate facts, inference, gaps and advice; cite sources and follow the output contract	—
`eval-admin-closeout-observer`: reconcile PM's final report, worker reports and the current authoritative evidence chain	✓
`controlled-emergence-observer`: inspect task relationships for probes, self-tasks, sandboxes and project-tree patterns	—
`eval-risk-gap-analysis`: compare expected and actual behavior and recommend risk severity and ownership	—
`eval-promotion-advice`: recommend findings for follow-up tasks, issue drafts or reusable knowledge	—

eval-statistical-analysis governs calculation and interpretation: establish the task tree and time window, normalize and deduplicate events, then compute metrics. Separate active execution from waiting and assignment completion from product QA success. Missing evidence is unknown; conflicting evidence is disputed. Scoring is optional and requires an explicit formula; a high score cannot erase incomplete work or insufficient evidence.

The system-observation path also injects the closeout evidence-review skill, but its scope remains controlled by the system_observation contract. It does not thereby produce a separate closeout report. The repository also contains an auxiliary fcop-eval-promotion workflow for subsequent classification and internal drafts; it is not in these three paths' required skill lists. Advice does not automatically create tasks, publish issues, or change lifecycle.

The EVAL record matrix: logs, retained assets, and factual assessment

Read this system in three layers: logging retains what happened; asset organization makes tasks, reports, evidence bundles and indexes traceable; independent factual assessment checks claims, identifies contradictions, explains uncertainty and recommends follow-up. CodeFlowMu and the Host produce most raw logs; they are not all generated by EVAL. EVAL's analysis then becomes another retained file asset that can itself be reviewed and reused.

Factual assessment is a sourced, challengeable judgment, not an automatic declaration of truth or ADMIN acceptance. The seven skills make the matrix operational through statistics, issue analysis, writing, delivery verification, emergence observation, risk analysis and retention advice.

Evidence layer	File or field	What it establishes
Recording start	`.codeflowmu/eval-recordings/<RUN-ID>.json`	Run identity, boundaries, task association and generation state
Run material	`raw/`, `artifacts/` and indexes under `research/evidence/benchmarks/.../<RUN-ID>/`	Material available to EVAL and later reviewers
Program collection	Collected run record, asset scan, `*-evidence-bundle.md`	What software gathered; not an independent agent conclusion
Independent analysis	EVAL Session, model provenance, `analysis_skill_ids` or `skill_ids`, and skill receipts	Which session received which instructions; injection proves loading, not analytical quality
Final report	Analysis and agent/session/run provenance under `fcop/internal/eval/`	Persisted independent analysis whose citations can be checked again

The EVAL agent reads evidence and returns analysis text. Runtime checks the required format, provenance and applicable skill receipts before persisting it. Program collection, agent analysis and final persistence are separate stages. A file appearing, a session ending, or a “generated” label alone is insufficient proof of a valid completed analysis. See the actual code excerpts in section 3.9.

CodeFlowMu provides both the execution evidence and a business workflow for another agent to question and analyze it. EVAL advises; PM remains responsible for delivery; ADMIN retains acceptance and follow-up decisions. This article additionally checks EVAL against the separate per-run backups. Three implemented report paths do not mean that every run successfully produced all three reports. Missing reports, failures, retries and mismatched material remain part of the evidence.

Implementation sources for this section are test commit cb590ce: codeflowmu-shell/src/eval-independent-analysis.ts for routing and required skills, eval-benchmark-recording.ts, packages/evaluator/eval-report-writer.js, and EvalObservationGenerator.ts. These explain the mechanism; source code alone does not prove a particular execution succeeded.

1.11 Why nine asset classes, and what does EVAL inspect?

The nine classes come from CodeFlowMu's implemented system-observation inventory. They let an evaluator compare how the same event appears in different records. They are not nine agents, nine scoring dimensions, or a universal taxonomy. The test version explicitly lists them in ASSETS_ANALYZED; business files such as TASK, REPORT and ISSUE enter verification through ledger associations, runtime material and report references.

Asset	Main records	Analytical use
1. Ledger	Registered tasks, role routes, parent relationships, states and business-record links	Reconstruct formal work and compare lifecycle with delivery
2. Runtime logs	Sessions, attempts, leases, tool calls/results and errors	Reconcile execution, retries, failures and state changes
3. Public thinking/progress	Emitted plans, explanations and tool activity	Compare contemporaneous claims with later behavior and scope changes
4. Usage	Collected request, token and usage records	Attribute consumption where run associations exist and identify missing coverage
5. Analytics	Aggregated events, counts and timing metrics	Check raw events, deduplication, denominators and windows
6. Internal EVAL	Previous observations, collection material, analysis and generation states	Check evaluation identity, provenance, persistence and mismatched material
7. Emergence log	Previously observed patterns, sources, risks and recommendations	Compare task relationships and track duplication, evolution or unsupported findings
8. Role views	Role-specific task lists and status projections	Compare views with ledger, lifecycle and the view contract
9. Shared knowledge	Shared rules, experience, knowledge and reusable material	Check relevance, age and opportunities for retention

Asset	Question to answer
1. Ledger	What did PM assign, what was formally delivered, and what remains open?
2. Runtime logs	Does a claimed action have a result? Did a finished session actually deliver a report?
3. Public thinking/progress	Did promised dispatch occur? When did the plan change? Hidden reasoning is not inferred
4. Usage	Which run incurred usage? Are retries or other sessions included? Without billing attribution, it is not exact task cost
5. Analytics	Are reported call counts and durations calculated consistently?
6. Internal EVAL	Is this completed agent analysis or only a collected summary? Does it belong to another run?
7. Emergence log	Was this pattern already recorded? What changed, and is it reusable?
8. Role views	Does “running” match formal state? Is an empty task list correct at this stage?
9. Shared knowledge	Was available knowledge unused, or is useful knowledge missing?

The value lies in cross-asset verification. A PM claim of tool success requires a result, not only a call-start log. A “running” display that conflicts with cancellation receipts and session state needs projection checks. Runtime, usage and analytics help distinguish requests, retries and duplicate event records.

The scan retains four observation fields per asset: status, key finding, risk/value, and evidence. Independent EVAL then analyzes contradictions, gaps and recommendations. “All nine classes scanned” establishes inventory coverage, not that all nine were verified healthy. CodeFlowMu makes logs useful analytical assets by enabling cross-checks rather than merely accumulating files.

1.12 Emergence observation: inspect collaboration patterns, not only errors

EVAL also looks for task origins and organizational structures worth recording. “Emergence” has a bounded implementation here: it is not a claim that a model suddenly acquired new intelligence, nor a label for ordinary errors.

Observation	Identification evidence	Analytical purpose
Controlled emergence	Role routing combined with explicit probe, self-task and sandbox bootstrap markers	Establish where test/probe tasks came from and whether they contaminate business task lists, review queues or counts; retain probe evidence without treating it as delivery
Project-tree emergence	Actual parent relationships and role routes within a thread, forming root → phase → execution	Observe project/phase organization, assess value against outcomes, and check wrong parents, cross-thread links, cycles or closed parents with open children

Ordinary delegation: ADMIN root → PM assigns DEV / OPS / QA checks
Project-tree pattern: ADMIN main task → phase task → specialist execution tasks

A title containing “Phase,” “project” or “probe,” or simply creating more children, does not establish either pattern. Detection examines origin, parent, thread_key, routing and relevant markers. Independent EVAL checks the interpretation and reports alternatives, confidence, risks and advice. controlled-emergence-observer is one of the seven skills; the companion project-tree-observer.js supplies structural evidence, not another mandatory report.

Emergence observation examines both risk and value. Probe tasks appearing in business views may mislead users; a useful phase structure may provide a reusable way to organize work. Structure alone does not prove successful delivery. The system observation report and retained emergence log support later review and knowledge retention. EVAL recommends action; it does not autonomously clean up tasks, archive them or change dispatch.

No detected emergence is also a result worth retaining. Normal ADMIN→PM→DEV/OPS/QA delegation in this inspection cannot be marketed as new emergence merely because the team collaborated. Any per-run claim requires that run's report and relationship evidence. CodeFlowMu can accumulate observations about how collaboration structures form and whether they help, alongside success and failure records.

Implementation sources are the test version's packages/evaluator/eval-report-writer.js, controlled-emergence-observer.js, and project-tree-observer.js. The asset classes are an implemented inventory; the questions above explain analytical uses and do not assert that every question was verified in every run.

2. Overall Results and Each Team

The results below were checked against formal assignments, reports, execution, and cancellation records captured through CodeFlowMu. We separate completion, inspection quality, and integration failure.

2.1 The six results

Scheme	Normal delivery	Valid worker reports
Codex	Yes	Three roles
Cursor	Yes	Three roles, plus one cancelled old QA task
DeepSeek	Yes	Three roles
Doubao	Yes	Three roles
Qwen	No	DEV only; PM submitted a blocked report
Kimi	No	No downstream roles

Scheme	Elapsed time	Score / 100
Codex	12m 20s	88
Cursor	20m 16s	87
DeepSeek	39m 21s	75
Doubao	21m 59s	66
Qwen	Terminated after 122m 10s	40
Kimi	Terminated after 31m 43s	18

Scheme	Main observation
Codex	Fast closure with qualified conclusions
Cursor	Authorized recovery followed by delivery
DeepSeek	Deeper collection, some incorrect interpretation
Doubao	Smooth workflow, overconfident report claims
Qwen	Useful diagnosis, ineffective closure
Kimi	Integration/session failure before teamwork

Weights: completion 25, result quality 30, efficiency 15, scope control 15, recovery 10, observability 5. Scores describe these runs, not universal foundation-model ability. Codex and Cursor's one-point difference places them in the same leading group. See the score data.

2.2 Completion is more than a final paragraph

Four teams delivered three valid specialist reports and a normal PM final report. Qwen's OPS content remained in a failed submission, QA produced no formal report, and Kimi created no child tasks. Forced archiving stopped a test; it did not complete the assignment.

Codex took fewer actions to reach a qualified conclusion. Cursor showed authorized recovery. DeepSeek's advantage over Doubao was mainly evidence and interpretation quality, not a different final task-state label. Qwen's 40 acknowledges diagnostic work as well as failed delivery; Kimi's 18 describes a non-delivering configuration. Neither score can be read as an intrinsic model score. This was one run per model configuration, not a blind or randomized benchmark.

2.3 Codex: fast delivery with explicit limits

The PM assigned FCoP/MCP to DEV, runtime to OPS, and skills/permissions to QA, without explicit dependencies. The first child task appeared around 1m 21s after the formal session began. Three worker reports and the PM summary covered all five requested areas.

Its strength was distinguishing static configuration from demonstrated capability and preserving unverified limitations. QA did not turn workflow completion into an unconditional product pass. The 12m 20s duration coexisted with 24/30 result quality. Some verification remained static, so this was not comprehensive functional certification.

2.4 Doubao: a smooth chain with weak final review

At about 1m 01s, PM began assigning DEV to MCP, OPS to runtime and permissions, and QA to FCoP/SKILLS. The three tasks had no explicit dependencies. Formal delivery took 21m 59s.

The quality deductions concern unsupported claims: 23 tools claimed but 22 listed, inconsistent skill-count scopes, a greater-than-95% match rate without a denominator, and treating visible schemas as proof of successful parameter validation. PM did not sufficiently filter worker overstatement. Completion 25/25 and quality 12/30 describe different aspects of this same run.

2.5 DeepSeek: deeper evidence, imperfect interpretation

PM created three role tasks without explicit dependencies, with the first at roughly 6m 29s and delivery after 39m 21s. Reports distinguished 48 skill references from 54 on-disk directories and on-demand injection. They included software probing and runtime endpoint checks.

However, the team misinterpreted the independently configured Cursor EVAL as a governance violation and recommended committing runtime governance files to the main repository. PM did not correct those interpretations. Quality 18/30 reflects stronger fact collection but insufficient interpretive review. Agreement among several roles does not make a claim true.

2.6 Kimi: the team never reached formal downstream work

Runtime recorded eight formal PM sessions ending as failed: seven error endings and a final cancellation. The first formal session received an encrypted_content content-type rejection. Six subsequent failure results contained overload and stream-disconnection messages. Tool-preparation records also showed readiness had not been achieved.

PM read resources and requested recovery, but created no formal child tasks or team reports. ADMIN stopped the run after 31m 43s. This establishes that the tested model/Host/adapter combination failed to deliver; it does not establish that Kimi intrinsically cannot use MCP or can never work through Codex. Its teamwork was not adequately exercised.

2.7 Qwen: investigation expanded without complete delivery

PM took 11m 46s to create the first child task, then assigned OPS, DEV, and QA, with QA referencing DEV. DEV delivered. OPS generated a complete report body, but its formal submission failed in transport. QA produced no formal report. PM submitted a blocked report and the run was terminated after 122m 10s.

The team encountered genuine transport and dispatch problems. Saying it found nothing would be inaccurate. Yet later probing, waking, pausing, and explanations did not produce effective resubmission or closure. Sparse early progress text and extensive later explanation also made timely supervision difficult. The 40-point result recognizes both diagnostic contributions and failure to deliver.

2.8 Cursor: authorized recovery left an inspectable history

DEV covered skills and development-side tools, QA covered FCoP and role permissions, and OPS covered runtime/MCP. QA's unavailable model caused four launch failures. PM requested recovery authorization; ADMIN approved. The old QA task was cancelled and new task 005 ran with a link to the old task. Once all three valid reports were present, PM completed delivery in 20m 16s.

English reading guide — original Chinese UI: the root awaits ADMIN acceptance. Specialist rows show DEV, the original QA task, OPS, and a replacement QA task. The last task title includes “rerun after model recovery”; its existence alone does not prove the original QA succeeded. Cancellation and rerun receipts establish that distinction.

Click the screenshot to inspect its original pixels.

Four visible child-task rows did not mean four successful worker deliveries. Old QA task 003 was cancelled; task 005 supplied the valid QA report. A file in a done lifecycle directory must still be interpreted using its actual decision.

This was not fully unattended recovery: human approval was required. The authorization, cancellation, and rerun records are precisely what make the recovery explainable. The score of 87 reflects near-Codex delivery quality and organization under a real configuration problem.

2.9 How the score should be read

If we judged only a done label, four teams would tie. If we judged only duration, unsupported conclusions would disappear. Completion, quality, efficiency, scope, recovery, and observability must be considered together.

Doubao lost quality points for unsupported proportions and overclaiming. DeepSeek lost them for incorrect governance interpretation accepted by PM. Qwen's completion 10/25 and efficiency 2/15 acknowledge partial delivery while recognizing the missing OPS/QA reports. Kimi's completion 0/25 records the absence of formal delivery, not a measured inability to write a good report.

Removing efficiency and normalizing the remaining 85 points gives Cursor about 88.2 and Codex about 87.1. Their order reverses but their grouping does not. A one-point difference in a single-run, judgment-based rubric is not statistically meaningful superiority.

The transparency concern was inconsistent public communication: Qwen had long early stretches of tool activity with little explanation, then much more text when stuck. Later explanation cannot restore the opportunity to supervise and stop earlier. The evidence does not establish an intention to hide activity.

3. What the Evidence Lets Us Analyze

This section traces PM assignments through TASKs, time through sessions and returns, quality through worker and PM reports, and EVAL conclusions back to original material. CodeFlowMu's linked records make “what differed, and why?” an inspectable question.

3.1 Compare the PM's entire delivery chain

The same task did not produce the same task graph. PM chose who worked first, who reviewed whom, and which relationships counted as dependencies.

PM	Assignment pattern	Actual delivery
Codex	DEV: FCoP/MCP; OPS: runtime; QA: skills/permissions; no explicit dependencies	Three role reports and PM summary
Doubao	DEV: MCP; OPS: runtime/permissions; QA: FCoP/SKILLS; no explicit dependencies	Complete team chain
DeepSeek	Three-role inspection without explicit dependencies	Team reports with useful runtime collection
Kimi	No formal child assignment; tool discovery, recovery, and session failures	No team delivery
Qwen	OPS inspection; DEV code-level diagnosis; QA verification referencing DEV	DEV report, failed OPS submission, missing QA report, blocked PM report
Cursor	Three specialist tasks, followed by cancellation and linked QA rerun	Three valid reports plus retained cancelled task

PM	PM review/summary assessment
Codex	Qualified conclusions; static checks not presented as full functional proof
Doubao	Did not adequately correct counts, scope, proportions, or validation claims
DeepSeek	Accepted incorrect interpretations of EVAL configuration and runtime files
Kimi	No final report to assess; organizational capability not sufficiently exercised
Qwen	Some diagnoses were corrected later; recovery did not yield closure
Cursor	Authorized recovery is evidenced; human intervention remains part of the result

PM report quality means checking coverage of all five areas, evidence for important claims, correction of worker contradictions, explicit verified/unverified/failure distinctions, and recommendations within the mandate. Concatenating worker reports is not enough.

Kimi's missing final report is “not assessable,” not “a badly written final report.” Qwen's existing blocked report should be judged on obstacle explanation and proposed recovery. The article's result-quality score is a whole-run dimension, not a separately calibrated PM-writing score.

Three independent tasks were not necessarily superficial. Codex and Doubao divided the same scope differently; either can be reasonable when evidence and coverage are clear. Qwen chose a heavier diagnostic route, beginning delegation later and making QA reference DEV. Time before the first assignment is itself an organizational choice.

Reference semantics matter. A reference should supply context without automatically blocking work. This run exposed a dispatch gate that treated that relationship inconsistently. That is a system issue. PM still has to judge whether the added relationship is needed and how to adapt when it obstructs a bounded inspection.

3.2 Visible configuration is not demonstrated capability

Resource visibility, schema visibility, actual tool execution, and successful business delivery are different levels of evidence. One cannot substitute for the next.

Codex retained limitations and a partial QA conclusion. Doubao treated schema listing as validation and mixed skill scopes. DeepSeek made useful distinctions in its collection but then misread the independent EVAL setup. PM review must test the relationship between each claim and its source, not reward file volume.

Kimi never reached a comparable complete team chain. Its actual resource reads and recovery calls matter, but so do the request rejection and interrupted sessions. The records support failure of this configuration, not a universal inability to understand MCP.

3.3 Did longer runs buy more useful evidence?

Codex's fastest delivery still had materials corresponding to all five areas. Cursor's 20m 16s included authorized recovery. DeepSeek's additional checks had value, but its report also contained a wrong governance interpretation. Qwen's two hours combined real infrastructure obstacles with ineffective later investigation.

The four completed runs had a median of about 21m 08s. A 20–30 minute budget is a reasonable starting hypothesis for a future bounded inspection on this environment. It was not a limit imposed on these runs, so it cannot be retroactively treated as a contract violation.

3.4 Efficiency is evidence value per unit of effort

Among completed runs, Codex's speed did not accompany a lower quality score. DeepSeek spent about 27 extra minutes and collected additional runtime evidence, but did not produce a more trustworthy overall conclusion.

Elapsed time includes preparation, pre-delegation checking, the longest worker path, retries, PM review, and summary. Stages overlap; role durations cannot simply be added. Cursor also includes authorization waiting. The evidence does not support a precise allocation such as “X% model delay, Y% platform delay.”

More issue claims must first be screened for false positives. Unsupported proportions, old snapshots, and incorrect governance explanations create review work. Issue counts and report length are not substitutes for useful findings.

Cost comparisons also need boundaries. Qwen's roughly CNY75.56 came from a daily screenshot. DeepSeek's CNY15.6486 export included other requests in its hourly aggregation. Subscription channels do not have zero cost. This evidence supports a time-efficiency assessment, but not a precise cost-per-qualified-report ranking.

3.5 Where Qwen's two hours went

QA was created at 17:09:51 and started at 18:02:15: 52m 24s waiting. The declared relationship was informational_reference, yet a dispatch gate required successful DEV closure. The historical source and runtime trace support this issue.

After QA started, it ran for nearly another 50 minutes without formal delivery. The earlier dependency no longer explains that entire delay. OPS called write_report with complete content, but received Transport closed; no formal report was saved. PM then probed, woke, and paused repeatedly without an effective resubmission path, ultimately reporting blocked.

Some deeper claims should be withdrawn. The alleged “ghost running QA” used a topology snapshot from before QA started. The allegedly random limit failures compared string and integer arguments. The root cause of short-name instability was not demonstrated under equivalent conditions. Genuine issues, hypotheses, and mistaken interpretations must remain separate.

3.6 Cursor's recovery was a governed sequence

System initialization had occurred, but the QA seat was misconfigured with qwen3.8-max, unavailable through Cursor SDK. OPS identified a model-availability problem rather than an MCP or lease failure. PM requested authorization at 21:51:56; ADMIN approved around 21:55.

The old QA task was cancelled. A new task linked by rerun_of started at 22:00:12 and produced its report at 22:01:46. PM's successful final submission followed at 22:04:27. These relationships preserve the failed attempt rather than overwriting it as success.

Recovery of an authorized execution prerequisite is different from unauthorized scope expansion. In the September 8 comparison, the first Codex run expanded inspection into code repair and was stopped. That was a separate historical run. The September 9 Codex run did not repeat the behavior; this does not prove the earlier event was caused by another AI's old records.

3.7 Does the Codex integration explain failure?

Both models with incomplete runs used the Codex framework. Integration is therefore part of the causal analysis. The evaluated object is the model, provider API, adapter, Host, and CodeFlowMu working together—not a foundation model in isolation.

Scheme	Evidence	Supported conclusion
Kimi	One explicit rejection of `encrypted_content`; six later disconnections carrying an overload message; no downstream tasks	A content-compatibility error and runtime failures occurred. Team capability was not sufficiently exercised to infer weak intrinsic ability
Qwen	Created tasks, used tools, delivered DEV work; QA dispatch waiting, OPS submission failure, old-snapshot misinterpretation, extended investigation without closure	Integration could operate, but the delivery chain had faults and PM handling was insufficient. Compatibility does not explain everything

Kimi's overload messages are not themselves compatibility errors. Qwen's dispatch gate issue belongs to this system workflow and is not automatically attributable to Codex or Qwen.

Codex, Doubao, and DeepSeek completed through the same overall framework, so “uses Codex” is not sufficient to explain failure. Provider-specific request content, tool representation, returned results, and recovery behavior matter. Cursor's successful SDK run changed both model and framework, so it is not a controlled demonstration that moving Kimi or Qwen to another Host would fix them.

The tested source configured custom providers for Responses and disabled Codex reasoning-summary metadata support. The Doubao Ark bridge also normalized request fields and changed tool exposure. That Ark-specific behavior should not be attributed to every provider. Identical framework names do not guarantee identical effective tool catalogs, requests, or public output.

Disabling a summary does not establish that a model stopped reasoning or could not provide public progress. Qwen's sparse early explanation and later verbosity require comparison of raw assistant output with Host and UI events. The observed inconsistency harms supervision; deliberate concealment is not established.

To separate model ability from integration effects, repeat the same model and task on another verified executor with matched tool semantics and budgets. A website chat is not an equivalent control. These six runs do not quantify an integration penalty.

3.8 The evaluator also needs checking

A uniform evaluator cannot compensate for incorrect inputs. Some task-record analyses described the current run but linked to raw packages containing morning Codex sessions, role routes, and report hashes. Initialization reused CUSTOM identifiers, creating a risk of mixing windows. coverage=complete did not establish identity correctness. This article used separate run backups rather than copying disputed counts.

There was also a report-generation problem: replay showed section extraction treating child headings as the end of a parent section, making an otherwise populated report appear empty. The UI sometimes displayed completed as a failure reason. A later successful Cursor generation proved that one output was accepted, not that the defect had been fixed.

Billing has an analogous identity problem. DeepSeek's export contained 168 requests, about 16.22 million tokens, and CNY15.6486, with hourly aggregation including identity chat. Qwen's daily screenshot showed about CNY75.56 without task-isolated detail. Unknown cost must not be replaced with zero or an invented precise ranking.

3.9 EVAL record reports and observation reports

EVAL used Cursor throughout. The following comparison focuses on system observations and task-run records; a separate closeout path checks PM’s final report and its evidence chain. The three paths are distinguished in section 1.10. Programmatic collection and independent analysis are separate stages. A generated collection record is not automatically a finished EVAL judgment.

01 / Collect the run material

The collector receives the project root and run ID. Collecting material is distinct from completing independent analysis.

  const child = spawn(
    process.execPath,
    ["packages/evaluator/eval-benchmark-record.cjs", "--project-root", projectRoot, "--run-id", runId],
    { cwd: projectRoot, detached: true, stdio: "ignore" },
  );

Source excerpt: codeflowmu-shell/src/eval-benchmark-recording.ts, lines 592–596, test commit cb590ce35686cb1980e3c89a7d68bd0cfbeb825a.

02 / Start a separate EVAL session

The target report, analysis kind and root task are bound to an EVAL session. The tested EVAL configuration was Cursor / auto-smart in every run.

      const handle = await runtime.sessionManager.startSession(
        agentId,
        sessionTaskId,
        {
          text: skillInjection.prompt,
          maxToolRounds: DEFAULT_SESSION_MAX_TOOL_ROUNDS,
          uiLang: readPanelUiLang(getProjectRoot()),
          context: {
            eval_observation: true,
            eval_analysis: true,
            analysis_kind: request.analysisKind,
            analysis_target_path: request.reportPath,
            root_task_id: request.mainTaskId,
            session_kind: "CHAT_BOUND",
          },
        },
      );

Source excerpt: codeflowmu-shell/src/web-panel.ts, lines 17863–17879, test commit cb590ce35686cb1980e3c89a7d68bd0cfbeb825a.

03 / Validate the analysis before accepting it

A completed session does not automatically constitute a valid report. This excerpt rejects malformed analysis; the surrounding function also checks provenance and required skill receipts.

  const contentErrors = validateEvalAssistantText(content, input.analysisKind);
  if (contentErrors.length) {
    throw new Error(`EVAL_ANALYSIS_FORMAT_INVALID: ${contentErrors.join(",")}`);
  }

Source excerpt: codeflowmu-shell/src/eval-independent-analysis.ts, lines 286–289, test commit cb590ce35686cb1980e3c89a7d68bd0cfbeb825a.

04 / Persist the report as UTF-8

After analysis and provenance are assembled, a temporary UTF-8 file replaces the target report. This code explains the mechanism; it does not prove that any particular run passed the checks.

  const temporary = `${absolute}.analysis-${process.pid}-${Date.now()}.tmp`;
  writeFileSync(temporary, raw, "utf8");
  renameSync(temporary, absolute);
  return absolute;

Source excerpt: codeflowmu-shell/src/eval-independent-analysis.ts, lines 393–396, test commit cb590ce35686cb1980e3c89a7d68bd0cfbeb825a.

The risk labels below come from independent analysis sections. High risk can concern evidence identity rather than poor team delivery; absent reports do not mean low risk.

Scheme	Panel observation	Task-record analysis
Codex	Low	Medium
Doubao	Medium	High
DeepSeek	Low	High
Kimi	Final report not obtained	Final report not obtained
Qwen	Medium	High
Cursor	Medium	High

Scheme	Interpretation
Codex	Completed inspection with remaining evidence gaps; partial observation, not unconditional product pass
Doubao	Current delivery existed, but frozen materials were disputed
DeepSeek	Task/report inventory broadly consistent; raw package identity disputed
Kimi	Underlying recording and execution evidence still exist
Qwen	Business blocked and frozen-material identity disputed
Cursor	Live recovery traceable, but linked task-record materials disputed

Doubao and Cursor EVAL analyses corrected a scanner's false alarm: after the root task moved to ADMIN acceptance, an empty PM todo view could be correct. The completed run must not be labelled missing solely from that projection.

Kimi had records even without a completed final EVAL pair. Its archive contains a 17,488-byte .codeflowmu/eval-recordings/CUSTOM-20260909-001.json, begun at 16:08:52 Beijing time, with state: recording and generation_attempts: 0 in that snapshot. Task/chat process files and Runtime events are also preserved. This establishes a recording entry, not successful final report generation. We did not fill the gap using another run's reports.

English reading guide — original Chinese UI: “生成任务记录” means “Generate task-run record”; “已生成” means “Generated”; “报告已生成” identifies the saved report; “查看记录报告” opens it. The background lists the standard task-run record and EVAL panel scan.

Click the screenshot to inspect its original pixels.

The screenshot shows a successful later Cursor report. Saving a report is one check; matching its content and identity to the run is another.

3.10 EVAL is useful, but not the final truth

The article asks both whether the team delivered and whether its evidence is reliable. EVAL is valuable because it can challenge a convincing report. Its own claims also need review.

Cursor's score of 87 and high task-record risk are not contradictory measures of the same thing. The score uses the corresponding independent run backup and recovery chain. The risk addresses the associated record package's identity problem. We did not use disputed raw counts as if they belonged to the current run.

EVAL's corrections were often more useful than adding issues: it rejected “empty PM todo means lost work,” and distinguished actual Qwen transport failure from an asserted permanent dependency deadlock. DeepSeek's claim that the independent Cursor EVAL was a violation also needed correction: that configuration was intentional.

CodeFlowMu's product value is that successful delivery, authorized recovery, disagreement, and defects can all be investigated from retained records. Reliable run identity, correct freezing, and truthful generation states are priorities for improvement.

3.11 Fact checking: return a claim to its source

Claim	Evidence compared	Finding
“OPS finished the report”	Complete call body, returned status, formal file and receipt	Qwen generated the body but failed submission
“PM todo is empty, so work was lost”	Root state, ADMIN acceptance view, projection contract	EVAL corrected the scanner's inference
“QA is running but has no session”	Snapshot time, QA launch time, session record	A pre-launch snapshot cannot prove ghost execution
“This raw package belongs to this run”	Model, time window, Session, routes, hashes	Several same-named packages contained earlier Codex material

Identify the same run and attempt before asking what a record proves. Successful transport, successful command execution, and successful business delivery are different outcomes. Missing evidence remains unknown; conflicting evidence remains disputed until resolved.

No usable external third-party fact source was configured in this test. These checks relied on local execution evidence. They are not an external database certification or an automatic endorsement from a standards authority.

3.12 Diagnosis: make “stuck” a specific problem

Diagnosis should explain what happened, where the evidence is, and which layer needs investigation next. CodeFlowMu linked tasks, sessions, tool results, and logs to formal work, rather than leaving diagnosis as a conversational impression.

Layer	Observed fact	Diagnostic implication
Configuration	Cursor QA used an unavailable model	Authorized correction, cancellation, and a linked rerun restored delivery
Submission	Qwen OPS returned `Transport closed` after sending complete content	Confirmed submission failure; inspect recoverable delivery rather than blame only the model
Dispatch	Qwen QA's reference relationship still waited for DEV at one gate	Explains initial waiting, not all later non-delivery
Integration/session	Kimi content rejection and interrupted sessions	Validate the request and endpoint path before judging full team ability
EVAL output	Material collected but extraction, validation, or UI status problematic	Separate collection, analysis, and final file write

Useful diagnosis narrows the problem and supports recovery. Cursor supplied an end-to-end recovery example. Qwen supplied a case where actual issues were found but diagnostic scope was not controlled. Independent evaluation raises questions, fact checking tests their basis, and diagnosis identifies causes and possible next steps. PM and ADMIN retain responsibility for decisions.

3.13 The two incomplete runs, with precise evidence

Kimi did not establish downstream teamwork. Qwen established it but failed to finish delivery. ADMIN termination was the ending action, not the sole explanation for what preceded it.

For Kimi, the archived Runtime file contains an explicit error at 16:15:10: invalid_request_error: responses: unknown content part type: "encrypted_content". Six later failure results, from 16:19:18 to 16:33:12, contain responseStreamDisconnected and The engine is currently overloaded, please try again later.

The eight formal failed endings consist of seven error endings and one final cancellation. Four recovery calls returned refresh_queued with tools_ready:false. Resources and commands were used, but no formal child TASK or REPORT resulted. In the corresponding archived sdk.result records for Codex, Doubao, DeepSeek, and Qwen, these two specific signatures were not found. That comparison does not imply their tool paths were fault-free.

The content rejection is direct compatibility evidence. The overload text is a returned service signal, not an independent measurement of actual server load. We still cannot identify which component introduced or retained the unsupported content, the precise origin of the overload message, or whether the tool-readiness issue shares the same cause. See the timestamped source evidence.

For Qwen, the reference dependency's gate explains 52m 24s of waiting, and OPS transport failure explains a real delivery obstacle. QA then ran nearly 50 minutes without a formal report. Continued investigation included old-snapshot and argument-comparison mistakes. These support criticism of PM recovery and scope control, but do not quantify the share of blame attributable to each component.

Future validation should differ: Kimi first needs a minimal formal tool-discovery, child-task, and report chain; Qwen needs validated dispatch/submission recovery followed by a bounded inspection. These are proposed checks, not fixes already completed.

4. Conclusions: What CodeFlowMu Made Possible

4.1 The system made the comparison evidence-based

The demonstrated value is turning a natural-language request into teamwork that can be assigned, tracked, checked, and handed over. CodeFlowMu supplied operational visibility and management controls. FCoP made assignments and deliverables persistent. EVAL provided another opportunity to challenge conclusions.

Retained system evidence	Question it answers	Result in this test
TASKs and relationships	How did PM divide the work?	Distinguished parallel inspection, reference dependencies, and linked reruns
Sessions, returns, timestamps	What actually ran, and where did it fail?	Identified Kimi's content rejection and overload/disconnection sequence
REPORTs and receipts	Was content actually delivered?	Avoided counting Qwen OPS text as formal delivery
Worker reports, PM summary, reviews	Did PM correct unsupported claims?	Exposed Doubao's count/proportion/validation problems
Approvals, cancellation, reruns	How was recovery authorized and executed?	Reconstructed Cursor's recovery without erasing the failed QA attempt
EVAL records, reports, status	What was independently assessed, and what was missing?	Preserved disputes and allowed Kimi diagnosis from underlying records

These judgments depend on evidence captured and retained during CodeFlowMu operation. Host/tool errors are recorded as returned; FCoP artifacts express formal collaboration; CodeFlowMu connects them to the business workflow. The exported backups and subsequent checks then turn records into analysis. This is not a story reconstructed from agent memory or from a green status label.

System defects remain in the article because the same records make them inspectable. Success has delivery evidence; failure has diagnostic traces; recovery has an authorization history; evaluation has sources. Scores and diagnoses are EVAL and analytical judgments, not automatic business decisions made by the system.

4.2 Overall assessment of the six models

For this inspection, Codex is the first choice for routine execution; Cursor belongs in the same group where authorized intervention is available. Among the domestic-provider models, DeepSeek is the first candidate for further testing, followed by Doubao. Qwen needs bounded task-control testing; Kimi needs integration validation first.

Codex and Cursor both scored 24/30 for result quality. Their advantage was keeping conclusions proportionate to evidence while reaching delivery. DeepSeek and Doubao showed they could organize the workflow, but PM review did not consistently filter incorrect interpretation or overstatement. Qwen and Kimi require different diagnoses rather than a shared “bad model” label.

4.3 Capability, reliable delivery, and review are separate thresholds

Doubao and DeepSeek demonstrated task decomposition, tool use, and formal team reporting through this framework. That does not establish that every domestic model is ready for an unsupervised PM role, nor does the failed pair prove that domestic models lack the underlying ability. Running successfully, reporting truthfully, and recovering effectively are distinct thresholds.

CodeFlowMu improvements should prioritize reference-dependency semantics, recoverable report submission, cross-run identity, and EVAL format handling, followed by stale display and progress-text issues. Traceability has been demonstrated; perfect calibration has not.

The original “inspect and report” wording did not authorize code repair. Production requests can further specify read-only scope, permitted formal assignments/reports, a budget, and when to stop investigating. Clearer language reduces ambiguity but does not replace PM judgment. A separate bounded-text test should not be mixed with the original wording and presented as an unexplained model improvement.

The scores remain 88, 87, 75, 66, 40, and 18 for these integrated runs. Foundation-model ability and integration loss were not separately measured.

4.4 A reliable team also knows when to stop

Inspection does not need to eliminate every defect. Reliable findings, supporting evidence, and sensible next steps can complete the assignment; repair needs its own authorization.

A future A/B protocol could retain the original task in one arm and explicitly bound read-only work and time in the other. Rotate the order, repeat each model configuration at least three times, and preserve task records, system observations, any triggered closeout observations, and failed drafts. Exporting/checking old evidence and verifying the initialized next environment solve different problems.

A common Git commit is not a full machine snapshot. Provider, adapter, effective tool catalog, network, task graph, and interventions still differ. Verify actual role sessions after configuration changes, not merely dropdown labels.

4.5 Methods, sources, and limits

Evidence was retained by run and archive batch: TASKs, REPORTs, sessions, tool receipts, chat and public progress, approvals, issues, logs, and EVAL. Qwen and Cursor have late supplements. ZIP integrity, file lists, and hashes were checked before comparison. Complete private logs are not published here.

Timing ends at successful formal final-report submission; termination is separately labelled. The 100-point rubric is a transparent judgment-based assessment with six weights, not a statistical confidence interval. One formal sample per scheme, fixed order, different provider paths, and Cursor's intervention limit generalization. Initialization was shared, but every cache, skill, historical reference, and external condition was not proven identical.

A reference answer for the inspection should cover all five areas with scope, reproducible evidence, conclusions, unknowns, worker receipts, and a PM summary. It must be tied to the actual version and configuration, not a permanent answer independent of system state.

The public package contains the article, figures, selected evidence, scoring/timing data, and source explanations. The concept cover is AI-generated, not a scene photograph. Screenshots are original evidence; diagrams explain mechanisms and are not substitutes for execution. All analytical charts and mechanism diagrams have English editions. The original article provides vector figures; this forum edition uses PNG exports of those same figures. Original screenshots retain the Chinese interface as captured, with English reading guides; literal evidence excerpts preserve their source wording. Supporting source documents are labelled by language in the public source guide.

Public source guide · Evidence inventory (Chinese) · Actual excerpts · EVAL source and checks · Kimi error evidence · EVAL code excerpts · Scores · Timing · Stage timeline

This public article and its supporting files are published separately from the private CodeFlowMu source repository. No access to the private repository is required to read the article or the disclosed evidence excerpts.

Original English article · 简体中文

Research repository

When Cleanup Resets the Retry Budget

joinwell52 — Thu, 10 Sep 2026 03:31:47 +0000

When Cleanup Resets the Retry Budget

A retry limiter can record an attempt correctly and still lose control before the next call. An Orca recovery-module experiment exposed the reason: display cleanup erased a terminal's recovery history while that terminal remained eligible for another remount. The useful review target is the cleanup predicate.

Adapted from the complete English research article.

Two indexes disagree about one object

Orca is a desktop tool for organizing terminals and Agent work. PR #19745 describes a Windows crash in which eight tabs reportedly remounted 8,878 times in roughly 122.4 seconds. We did not obtain the original crash bundle; those numbers are the author's incident report, not our measurements.

The public code exposes a mechanism worth testing. Terminal rows live in one index, while unified UI tabs live in another. They usually correspond but can diverge.

Remounting consults the terminal-row index. The old budget-release path consulted the unified UI index instead. The same tab could consequently receive two incompatible answers: it existed for the function performing a remount, but appeared absent to the cleanup function deciding whether to erase its recovery history.

Each lookup returned a definite answer. Their combination allowed recovery to discard the evidence of its own activity.

Test the causal path without claiming a desktop reproduction

We pinned the base and proposed head revisions and loaded the complete recovery and lookup modules. The original budget calculation, instance disposal, remount and generation-update logic remained intact. We supplied fixtures for store plumbing, time, timers, PTY and logging. We did not start Electron or reproduce graphics-memory exhaustion.

Each cycle registered an instance, requested recovery and unregistered it. We then made the UI index miss a tab whose terminal row remained present. Two request schedules tested different constraints: 10 milliseconds between calls for the cooldown, and 16 seconds to move past the cooldown while remaining inside the cumulative budget window.

Scenario	Requests	Base remounts	Proposed-head remounts
Both indexes agree; 10ms spacing	200	1	1
Terminal row present, UI entry absent; 10ms spacing	200	200	1
Same divergence; 16s spacing	10	10	3

Both rounds agreed. The first row shows that the old limiter was not universally ineffective. It worked when disposal could still see the tab. The second demonstrates history being reset under index divergence. The third separates the cumulative cap from the cooldown: even with requests spaced apart, the old code never retained enough history to reach its limit.

These counts describe successful remounts in the fixture's in-memory store, not production crashes.

Share the predicate, not another counter

The proposed fix does not lower the cap or introduce a second limiter. It makes the question “can this terminal still be remounted?” and the question “has this terminal gone away?” rely on the same terminal-row lookup.

Replacing a display instance does not end the lifetime of the terminal object that owns the recovery budget. If that object can still produce another recovery effect, a missing entry in a different UI projection should not erase the history constraining it.

This is a bounded correction. The two indexes can still disagree elsewhere. The patch chooses the appropriate source of identity for this budget; it does not establish system-wide index consistency.

Verify legitimate recovery still works

Merely proving that a storm stops is insufficient: disabling every recovery would also satisfy that test. We retained three controls.

Control	Result on both revisions
Fixture removes the terminal row, disposes the instance, then recreates the same ID	Two requests produce two remounts
Requests at 0, 16, 32, 48 and 300.001 seconds	First three accepted, fourth constrained, final request accepted: four remounts
Replay the old generation after one successful recovery	Two requests produce one remount

The close control mutates the fixture's terminal rows; it is not a real UI close action. It checks the budget-release branch. The expiry control checks that the cap does not become permanent, while the stale-generation control checks that delayed requests cannot continue acting on a replaced display instance.

Together, these controls distinguish preserving necessary history from freezing recovery altogether.

A budget depends on evidence surviving cleanup

The tempting response to a recovery storm is another limit. This case suggests inspecting the existing limit's release conditions first. Who can erase its history? Does that decision rely on the object performing the action, or on a UI projection or cache that can temporarily lose sight of it?

The same question applies to retry counts, failure lockouts and spending budgets. An accurate timestamp is ineffective if the action's own cleanup path deletes it before the next admission decision.

Not every budget needs permanent persistence. Its lifetime does, however, need to cover the object it constrains, and its release predicate needs to agree with the execution predicate. The disappearance of a UI instance does not necessarily end a backing object's history.

The reusable finding is simple: a protection mechanism must both record an action correctly and preserve that record until the next decision that depends on it. Cleanup deserves the same scrutiny as admission.

A review you can apply

Test both the admission path and every history-deletion path. Keep controls for real object disposal, window expiry and stale requests so that stopping a storm does not silently disable valid recovery.

Full English edition · 中文版本 · Methods and saved observations

Research repository

Test the Scope of Your Concurrency Limit

joinwell52 — Thu, 10 Sep 2026 03:30:08 +0000

Test the Scope of Your Concurrency Limit

Two event loops can each respect a concurrency limit of one while running two tasks in total. Our controlled AG2 experiment separates that valid local behavior from a real cache-replacement defect that let one loop admit too many tasks. The distinction changes the assertion a resource-budget test needs.

Adapted from the complete English research article.

Reproducing the actual defect

AG2 is an open-source framework for building and coordinating Agents. PR #3243 changes the semaphore cache used when admitting subtasks. A semaphore supplies a limited number of slots: later coroutines wait when all slots are occupied, and a completed task releases its slot.

The earlier method stored one semaphore and its owning event loop on the Agent. When another loop arrived, it replaced the cached semaphore. Threads alternating between loops could keep replacing that reference while earlier tasks still occupied older semaphores.

We pinned the base and head source and executed the original _spawn_subtask method. The subtask body was a waiting counter, not a model call. Rather than relying on random scheduling, we arranged an explicit interleaving:

A1 enters on loop A and holds a slot.
B1 enters on loop B in another real OS thread.
While both remain active, A requests A2.
We record local and aggregate peaks, then release the tasks.

On the old method, A2 obtained a newly created slot. Loop A now had two active subtasks and the experiment had three overall. This violates even the narrower per-loop interpretation of a limit of one.

Which peak becomes one after the fix?

The proposed implementation keeps a semaphore for each running event loop and protects lookup and creation with a thread lock. B no longer replaces A's entry. Under the same interleaving, A2 waits for A1 to release its slot.

Revision and execution shape	Loop A peak	Loop B peak	Aggregate peak
Base, one loop	1	0	1
Base, two loops	2	1	3
Proposed head, one loop	1	0	1
Proposed head, two loops	1	1	2

We ran each configuration five times per round, across two rounds: forty controlled trials with identical outcomes and no exceptions. Recording both local and aggregate peaks makes the result precise. The proposed method repaired the per-loop limit; two loops still ran two subtasks in aggregate.

These are in-flight synthetic waiting subtasks, not measurements of model requests, throughput or cost.

The upstream regression test explicitly targets the per-loop contract. An aggregate peak of two should therefore not be called a failed fix. It identifies a different guarantee that this patch does not provide.

Turn the number into an acceptance contract

A limit intended to protect a real resource needs at least four definitions.

The first is its scope: loop, Agent, process, host or tenant. The second is its admission authority: which object or service can issue slots, and whether all relevant executors share it. The third is its lifetime: what task completion, process exit, lease expiry or manual cleanup means for an occupied slot. The fourth is its uncertainty policy: losing sight of an old task does not establish that the resource it held is free.

A user may select one to prevent two expensive tasks from using the same account simultaneously. A correct semaphore in each loop could still fail to satisfy that product requirement. Conversely, a product deliberately promising one task per loop should not silently impose cross-domain serialization in the name of safety and remove intended parallelism.

The design starts with the contract, followed by the appropriate lock, shared counter or lease. A lock around creation protects that operation. It does not automatically turn separate counters into a shared resource budget.

What this experiment establishes

We did not run a complete AG2 Agent, toolchain or model service, and did not test multiple processes, hosts or tenants. Initialization was a version-matched fixture; the admission method itself came from pinned upstream source. The PR remained open at the time of our inspection.

The useful acceptance question is therefore not only “did the peak exceed the cap?” It is also “within which domain did we count that peak?” Without that domain, the limit is difficult both to implement and to review fairly.

A review you can apply

Add separate assertions for each loop's peak and the aggregate peak. Choose which one represents the product contract before deciding whether two concurrent tasks are a defect.

Full English edition · 中文版本 · Methods and saved observations

Research repository

Zero Credential Files Is Not a Secret-Access Test

joinwell52 — Thu, 10 Sep 2026 03:30:06 +0000

Zero Credential Files Is Not a Secret-Access Test

A test asserting that no credential file was created checks one delivery route. In an OpenHands source-seam experiment, file callbacks fell from four to zero while all four synthetic credential blobs still reached a real child process as environment variables. Here is how to test the recipient's actual inputs without overstating what a provider-scoping patch promises.

Adapted from the complete English research article.

A useful fix with a narrower contract

OpenHands Agent SDK can launch different agent programs through ACP, a protocol connecting an agent client to an execution program. Those programs use different authentication mechanisms, including environment variables and credential files.

PR #4927 narrows file-secret handling by provider ownership. The default configuration combines multiple providers' file rules; the proposed change avoids materializing unrelated providers' files when starting one particular program. That is a useful correction to credential representation and directory configuration.

File rules, however, are not a secret allow-list. The startup path also exports registry secrets as ordinary environment variables, excluding those handled as files. Narrowing the file set narrows this exclusion set too. A value removed from one route can consequently reappear on the other.

This does not make the patch a failed implementation of its stated goal. It means reviewers should not add a stronger promise—“all other providers' secrets become inaccessible”—that the change does not establish.

Observe the recipient, not just the writer

We pinned the PR's base and head revisions and executed its original filtering methods, environment-conflict method, and contiguous environment-construction slice. Both revisions used registry metadata extracted from the head to isolate the handling change. Provider identity was fixture-supplied; actual command detection was not tested. All inputs were synthetic strings.

File materialization was a recording fixture: the file-side numbers below count callbacks, not authentication files actually written to disk. The assembled environment was passed to a real Python child process, which returned variable names only. It did not launch a model or attempt authentication.

Revision and recipient	File-side callbacks	Four credential-blob variables visible to child
Base, Claude	4	0
Proposed head, Claude	0	4
Base, Codex	4	0
Proposed head, Codex	1	3
Proposed head, unrecognized provider	4	0

Both rounds produced the same results. The four blobs correspond to registered file credentials for Codex, Gemini, Kimi and Pi. Codex retained its own file route while three other blobs became ordinary environment variables. An unrecognized provider retained every file specification. Those branches cannot all be summarized as “only this provider's credentials remain.”

The figure uses our local seam-test observations. It establishes delivery of synthetic values through the tested construction path, not their use by a real CLI or disclosure over a network.

An empty list can have an authentication exception

Draft PR #4931 addresses a different layer: profile-level restrictions on secret names received by a conversation. Filtering the registry can narrow both downstream delivery routes.

We placed that PR's original filtering functions before the earlier environment slice. This is explicitly a laboratory composition of two separate PRs, not a merged SDK release or a server integration test.

Given the same six synthetic inputs, the Claude ACP profile produced:

`secret_refs`	Names retained in the registry
`null`	All six supplied names
`[]`	The provider's `ANTHROPIC_API_KEY`
`[PROD_DB_URL]`	The database variable and provider authentication variable

The empty list removed business secrets while retaining an explicit authentication exception. The program still needs a way to authenticate. As a separate control, we executed only the ordinary OpenHands profile's empty-list filter and obtained an empty set; that control did not pass through ACP startup.

An interface therefore needs to explain what its empty list means. “No selected business secrets” and “this process receives no secrets at all” are materially different expectations.

The registry is not every input channel

The tested startup slice also merges the host environment. A separate synthetic marker placed on that route remained visible to the child in every ACP scenario, including the empty-list case.

This is not a production leak report. It shows that a registry filter cannot constrain an input that bypasses the registry. The draft explicitly leaves host-environment inheritance outside its scope.

A stronger acceptance test begins by defining the business secrets and authentication materials authorized for a provider and run, then observes the actual environment and file capabilities at the receiving process boundary. Representation changes should follow that selection. Unknown-provider behavior needs its own explicit policy as well.

Our experiment did not exercise the complete SDK, a real ACP handshake, authentication, or server request filtering. Its conclusion is narrower: a disappearing credential file is insufficient evidence of disappearing access. Trace every delivery route for the same secret and inspect the set the recipient actually receives.

A review you can apply

Before accepting an isolation change, enumerate file materialization, registry export and ambient environment inheritance separately. Assert the allowed values at the receiving process, then keep the CLI/authentication boundary explicit.

Full English edition · 中文版本 · Methods and saved observations

Research repository

The Tool Result Was Blocked. Why Had the Operation Already Run?

joinwell52 — Wed, 09 Sep 2026 07:39:59 +0000

One tool call produced an executed receipt followed by a blocked receipt. A screen showing only the last word could suggest that nothing happened.

Our fixture's counter said otherwise: the tool ran once. Its returned content was withheld afterward.

There is no contradiction. “Blocked” needs an object: the call, the returned information, or subsequent processing. Compressing these into one overall status can erase precisely the detail that governance added.

Permission to call is not permission to disclose

AG2 is an open-source agent application framework. PR #3240 proposes scanning tool output because an authorized call can still return sensitive information. Input checks cannot inspect data that has not yet been returned.

We fixed the then-unmerged commit bf363ef52853e7b918863dec2fe1e3902b811e6f and exercised its original middleware entry point. Sensitive-looking strings came from public test fixtures, not credentials.

In the enforced secret case, the tool counter reached one, the result became a governance error, and the returned object no longer contained the fixture secret. Receipts recorded execution and subsequent withholding.

That establishes behavior at this middleware boundary. The tool was a counter fixture; no business write, network effect, or actual model-context ingestion was tested.

Modes and containers change the boundary

We retained the upstream policy, scanning, rewriting, and receipt logic while replacing framework event shells. Every case passed through the original on_tool_execution; two independent processes produced the same observations.

Condition	Tool executions	Fixture content in returned object	Receipt sequence
ENFORCE, successful secret text	1	Withheld	executed → blocked
MONITOR, same text	1	Redacted	executed → executed
OBSERVE, same text	1	Preserved	executed
ENFORCE, secret in error object	1	Preserved	error
FLAG policy	1	Preserved, finding recorded	executed → executed
Secret in structured value	1	Withheld	executed → blocked
Secret in structured key	1	Preserved	executed
SSN in structured string value	1	Redacted	executed → executed

Figure 1. schematic of the saved observations discussed above. Arrows show the stated processing relationship, not a runtime screenshot. Source: accompanying experimental evidence.

OBSERVE and FLAG are explicit policy choices, not bypass discoveries. MONITOR converts a blocking finding into redaction, so “not enforced” does not always mean “untouched.”

An error is another output path

The original code scans successful text parts and structured string values. It returns error objects directly. Its mapping traversal visits values, not keys.

The same fixture secret therefore receives different treatment when moved from successful text into an error object or dictionary key. That establishes scan coverage, not confirmed leakage into a real model: other framework stages might serialize, sanitize, or restrict these objects.

A useful review matrix names the containers and modes explicitly. Otherwise a successful text test can quietly become an unsupported claim about every return path.

Receipts need a stage

The sequence executed → blocked accurately describes an executed tool whose result was withheld. A consumer that treats the final receipt as the whole action's status loses this distinction.

Conversely, executed → executed need not mean two executions. In our monitor and flag cases the counter remained one; the second receipt described output governance.

Receipt counts are not tool counts. A clearer representation would identify the call, the recorded stage, whether execution occurred, and whether returned information was permitted onward. This is a design recommendation, not an implemented universal receipt contract.

For a real effectful tool, withheld output would not undo the effect. That is an inference from the ordering, not a business transaction reproduced here. Whether to retry must depend on the operation's own identity, reconciliation, or idempotency contract.

The evidence guide preserves eight cases and the precise substitution boundary. Actual Agent integration, message serialization, model ingestion, and external effects remain untested.

The input boundary decides whether a tool may act. The output boundary decides what information may continue after it acts. Both matter because they answer different questions.

Original English article · 中文版本

Research Center · Research repository

Only the Note Changed. Why Did the Evaluation Pass Rate Flip?

joinwell52 — Wed, 09 Sep 2026 07:39:14 +0000

Two evaluation records shared an ID. One said pass; the other said fail. We exchanged their notes, a and b, without changing either outcome. The summary's pass rate moved from 100% to 0%.

This was an isolated experiment against a real aggregation function, not a production scoring incident. Its eight existing tests still passed, including the test requiring identical output when input order changes.

The implementation was deterministic. What it lacked was a reason to treat its chosen record as the authoritative result.

The test answered a narrower question

CodeFlowMu is a local multi-agent collaboration system we are developing. An evaluation helper introduced in an August change aggregates outcome records. Its change record explicitly describes resolving duplicate identities through a content comparison, avoiding dependence on arrival order.

That is a useful goal. Filesystem enumeration and asynchronous completion should not determine an evaluation. But a repeatable choice is not necessarily a justified one.

The conflict test submits the same records in both orders and checks equality. It does not require preservation of both claims or an explicit disputed result. Consistently discarding one side satisfies the assertions.

The missing contract concerns identity: does the same ID with different content represent a repeated observation, or a conflict?

How a note acquired authority

We imported the actual summarizeEvalOutcomes function and supplied synthetic records:

[
  {"id":"A","status":"pass","note":"a"},
  {"id":"A","status":"fail","note":"b"}
]

The function groups by identity, serializes each complete raw record with sorted keys, and chooses the lexicographically smaller representation. The note participates in that comparison before the status. Swapping the notes changes the selected result.

Nothing establishes that the selected record is newer, better supported, or an authorized correction. The final summary retains only ID, name, and status, concealing the competing claim.

Controls matter

We ran eight cases in two separate Node processes, also reversing every input. These are function outputs, not estimates of model reliability.

Synthetic condition	Retained	Duplicates	Output
Different IDs, one pass and one fail	2	0	50%
Same ID, identical pass records	1	1	100%
Same ID; pass note=a, fail note=b	1	1	100%
Same outcomes; notes exchanged	1	1	0%
Same ID, pass/fail without notes	1	1	0%
Same case_id, different run_id, no record ID	1	1	100%
Same runs with separate record IDs	2	0	50%
ID present, outcome missing	1	0	unknown=1; rate field 0

Figure 1. schematic of the saved observations discussed above. Arrows show the stated processing relationship, not a runtime screenshot. Source: accompanying experimental evidence.

Both rounds agreed. Identical replay is correctly deduplicated, so abandoning deduplication would miss the point. The issue is treating replay and contradictory claims as the same category.

The missing-outcome control is equally important: the function preserves unknown. Its rate includes that record in the denominator, producing zero; that does not establish a confirmed test failure.

A test case is not a test run

The helper accepts case_id as an identity fallback and does not include run_id. Two observations of one case can therefore collapse when distinct record IDs are absent. With explicit record IDs, both observations survive.

This is an input-contract finding, not proof that production repeated runs were lost. Our source search found the helper and its tests, but no production consumer in the searched JavaScript and TypeScript roots.

Before asking how many repeated measurements establish stability, check that repetitions remain separate observations. An awesome-auditable-ai curation commit carefully narrowed claims about repeated-query research rather than endorsing a universal repeat count. We cite that editorial decision; we did not reproduce those statistical methods.

What should enter development review

Three cases deserve distinct contracts: identical replay, conflicting content under one record identity, and separate runs of one test. Preserve conflicting sources and expose disagreement instead of selecting a business outcome by note order. “Failure wins” would still be an unsupported selection rule unless explicitly justified.

The local helper is now the subject of a development-review submission. Consumer reachability, compatibility, and whether to retain the helper come first; no production incident or completed fix is claimed.

The evidence covers one historical change record, the actual helper at c008d9db91a21136fc61a4f60314e22db395d5d2, and eight synthetic cases. See the accompanying evidence guide. Product-source disclosure remains separately reviewed.

Determinism tells us whether an answer repeats. Conflict preservation tells us whether a single answer is justified at all.

Original English article · 中文版本

Research Center · Research repository

The Check Passed. What Still Has to Hold When the Agent Acts?

joinwell52 — Tue, 08 Sep 2026 07:22:38 +0000

When a file operation is approved, its source contains A. Must it still contain A when the executor reads it?

That sounds like a filesystem detail, but agents routinely cross delays: approval, queuing, resource availability, and execution. If an object or condition changes along the way, what does an earlier successful check still permit?

A natural response is to check again, closer to execution. That is useful, but incomplete without another question: what was checked, and what can change between that check and the actual effect?

We compared external proposals for start admission and exact file updates with a controlled timing experiment using CodeFlowMu’s real approval service and file executor. We did not reproduce a production concurrency incident. The post-check change was explicitly injected by the research code.

1. “Before execution” can refer to different objects

Paperclip is an open-source system for organizing agent work and runs. Its still-open PR #12949 proposes capacity admission after a run is claimed but before adapter startup. Observation and enforcement modes have different effects. This remains proposed functionality, not a delivered guarantee.

A capacity decision about starting a run is not authorization for every later file operation inside that run. Those operations have their own objects and conditions.

GitHub MCP Server Draft PR #3232 operates at another level: a particular repository file change. MCP exposes external operations as agent tools; this proposal binds an exact text update to expected branch and file-object identifiers.

The distinction is not a ranking of which project is safer. One concerns a run’s startup conditions; the other concerns a concrete repository mutation. Calling both a “gate” can hide their different responsibilities.

2. Moving the source change across the check

CodeFlowMu is a local multi-agent collaboration system we are developing. Its controlled file-operation requests include workspace and task identity, plus source and destination snapshots. Before execution, the current request is rebuilt and its digest is compared with the approved request. The digest is a fingerprint of the structured input, not a lock on the underlying file.

Earlier work had already shown that a changed destination can invalidate approval. Here we asked a different question: does changing source contents at different times produce the same outcome?

Using real components at commit c008d9db91a21136fc61a4f60314e22db395d5d2, we ran two copying scenarios. Both began with an existing destination and explicitly allowed overwrite. Overwrite was therefore not the disputed condition; the difference was when the source changed relative to digest validation.

Comparison	When the source changed	Result, identical in both rounds	Filesystem effect
Before validation	After approval, before rebuilding and checking the current request	`APPROVAL_STALE`; execution callback entered zero times	Original destination preserved
After validation	Inside the research callback after approval validation, before calling the real executor	Recorded as succeeded; callback entered once	Destination received changed source contents

The intervention in the second row is important. We did not observe a real user winning a race. After the approval service entered its execution callback, research code changed a synthetic source file, then called the existing product copy function. That point was deliberately controlled by the probe; we have not established equivalent control through an online interface.

The first row supports an existing protection. The second establishes behavior of the tested function composition under an injected sequence. Neither establishes who could change the source in a deployment, which ingress permits the same interleaving, or its likelihood.

Figure 1. E4/E5 timing comparison, two rounds each, with overwrite allowed in both. E5 explicitly injects the change after validation; it is not a reproduced production race. Source: formal controlled-study records; see the evidence notes.

Open full-resolution figure

3. The digest was correct—for the input it described

The approval service receives a current request and compares its digest with the approved one. On a match it records execution as in progress, then invokes the executor callback. The copy executor subsequently reads the source path.

In the first scenario, rebuilding the request observes the changed source snapshot. Validation rejects it before callback entry.

In the second, validation has already completed. The executor opens the same path, but the bytes there have changed. The approved source snapshot identifies source-before; the destination contains source-after, with a different digest. Both formal rounds produced that result.

There is no need to suppose a hash failure. The digest correctly describes the checked representation; it cannot prevent the mutable object behind that representation from changing afterward.

This is the time relationship studied under TOCTOU—time of check to time of use. Naming that relationship does not establish an exploitable race in a production path. A snapshot describes facts; it does not freeze them by itself.

Likewise, publishing a destination through a temporary file and rename can address one class of write-publication problem. It does not independently establish that the source bytes are still the approved ones.

4. Preconditions need a corresponding commit rule

The GitHub proposal connects expected identifiers with commit construction and a non-forced branch update. That is more specific than a preflight warning. However, rejecting non-fast-forward updates is not proof of strict compare-and-swap behavior under every branch movement or rewind. The author also reports not running the complete Go suite in their environment; we performed no remote integration rerun. Its result already includes before/after identifiers. The downstream question is whether callers preserve and bind that receipt. Proposal and limitations

Paperclip raises a separate applicability question: reattaching work already started is not the same as admitting a new start. Capacity should also correspond to the provider and model actually selected. Those issues appear in the proposal’s review discussion; we did not independently rerun its recovery or database-concurrency scenarios.

These designs illuminate our experiment without validating it on our behalf. A Git branch update is not a local copy; capacity admission is not operation approval. The common question is narrower: do validation, execution, and commit still refer to the same object and conditions?

5. Investigate the remaining interval before building a framework

Adding another check is not automatically sufficient: a change can occur after that check too, and different resources need different mechanisms.

First establish:

What was approved? A path, exact bytes, a version, or a combination?
What does the effect consume? Bound data, or a fresh read of a mutable path or configuration?
What preserves the condition through commit? Exclusive ownership, a conditional update, an immutable object, or only a prior observation?
What happens when conditions change? Zero-effect refusal, partial effects, and insufficient evidence are different outcomes.

A real-ingress experiment should include unchanged input, changes before checking, and changes after checking. Record source/destination digests, actual invocations, and receipts. Legitimate operations must still work. A refusal should be checked against resulting bytes, not merely an error message.

Our present result supports further investigation of the registered call path and filesystem competition. It does not authorize a general transaction engine or establish a live race vulnerability. Where a host, storage system, or executor already provides conditional-commit guarantees, investigate reuse before duplicating them.

Reliable execution requires more than “we checked before acting.” It requires an explanation of why the checked conditions still hold when the effect occurs.

Evidence and limits

This article uses E4/E5, twice each: four observations. Together with the companion article’s E0–E3, they make one twelve-observation formal batch. The English evidence guide, de-identified observations, and Chinese guide are publicly provided with this article.

The pilot also used no-overwrite in the timing cases, mixing two questions. The formal rerun allowed overwrite in both, isolating the timing comparison. Pilot records were retained but excluded from formal counts.

This is a Windows study of real product components with injected ordering—not live HTTP, panel, agent, multi-process contention, or remote GitHub mutation. Checking the records is distinct from rerunning the product. The writing stage performed no additional product experiment, code modification, independent QA, or repair.

Original English article · 中文版

Research repository

The Request Said “Do Not Overwrite.” Why Was the File Replaced?

joinwell52 — Tue, 08 Sep 2026 07:22:30 +0000

We put a no-overwrite restriction into a file-operation request. The approval record preserved it.

The destination was replaced anyway.

This was a controlled experiment with synthetic files, not a customer data-loss incident. The useful surprise was the comparison: with overwrite=false, writing refused an existing destination, but copying and moving succeeded and replaced its contents.

The approval record alone suggested that the restriction was explicit. The completion receipt alone suggested that the operation worked. Only when we compared both with the resulting bytes did the question become unavoidable: which component actually enforced “do not overwrite”?

1. Following a parameter all the way to the file

CodeFlowMu is a local multi-agent collaboration system we are developing. It organizes work through task files, execution sessions, and evidence. Its controlled workspace operations construct a concrete request, obtain one-time approval, check the request at execution, and delegate to a filesystem executor.

Our question was not whether approval could be bypassed. It was whether a restriction already inside the approval chain retained its meaning at the final operation.

The experiment used the real request builder, approval service, and filesystem executor. Each scenario created synthetic files in a fresh isolated directory and received a research-only administrative approval through the service. We did not invoke the live panel or let an agent touch business files.

At source commit c008d9db91a21136fc61a4f60314e22db395d5d2, we ran four scenarios twice:

Scenario	Restriction	Result, identical in both rounds	Filesystem observation
Write to an absent destination	No overwrite	Succeeded	New contents written
Write to an existing destination	No overwrite	Refused; operation recorded as failed	Original contents preserved
Copy to an existing destination	No overwrite	Succeeded	Destination replaced; source retained
Move to an existing destination	No overwrite	Succeeded	Destination replaced; source moved

The second row is essential. The write executor already has a working protection; the copy and move results do not justify saying that every file operation ignores overwrite constraints. The first row matters too: refusing more operations is not inherently correct. A legitimate new file should still be created.

These eight observations establish a difference between tested interfaces, not a production incident rate.

Figure 1. Explanatory rendering of E0–E3, two rounds each, retaining both the legitimate-creation and refusal controls. This is not an experiment screenshot; Source: formal controlled-study records; see the evidence notes.

Open full-resolution figure

2. The restriction survived. Its enforcement did not follow every branch

We traced the parameter through the code. The shared workspace input includes overwrite; the builder puts it in the approval request, and reconstruction from the approval record preserves it. Both copy and move records retained false.

The result was therefore not caused by a model omitting the flag, or by transmission turning false into true.

The difference appears in the final executor:

The write branch checks whether the destination exists and overwrite is explicitly forbidden. It raises target_exists_and_overwrite_is_false and preserves the bytes.
The copy branch calls the file-copy operation without using that flag in the branch.
The move branch calls the rename operation without using it either.

On the tested Windows system, the latter two operations replaced the destination. We read the actual result; this is not an inference based only on function names.

An approval digest can establish that an execution request matches its approved representation. It cannot, by itself, establish that the executor implements every restriction in that representation. A faithfully stored field does not enforce itself.

We also distinguished callback entry from operation outcome. The refused write entered the callback, then its exception was recorded as a failed operation. Copy and move were recorded as successful. A research process exiting normally is not proof that every product contract passed acceptance.

3. A “create file” proposal raises a related question

OpenAI Agents SDK is a library for building agent applications. In its still-open PR #4893, an author reports a create operation replacing an existing file and proposes an absence check. We did not independently rerun that work.

That restriction comes from the operation’s definition; ours comes from an explicit request parameter. Both ask whether the effect promised to the caller is enforced where the file changes. Neither implies that all write operations should become create-only.

A separate, also-open PR #4894 addresses delimiters inside mount-option inputs. It is a reminder that validating a permission and preserving a parameter’s meaning are separate responsibilities.

External cases illuminate the question. They neither reproduce our result nor prove that our product shares their complete failure paths.

4. Fix the executor—or first narrow the interface?

The tempting next step is to add a condition. Before choosing that implementation, we need to settle the contract: which operations is no-overwrite supposed to constrain?

If the product contract applies it to write, copy, and move, the latter branches need to enforce it with an appropriate filesystem operation. If the guarantee must also hold under concurrency, sequential checks are insufficient evidence; the interval between checking and writing needs its own tests.

If the flag is write-only, copy and move should reject it as inapplicable, or use input types that prevent the ambiguity. Silently accepting it into an approval record invites callers to rely on a restriction that does nothing.

We confirmed reproducible behavior at the internal controlled-request boundary. Static inspection also found the executor registrations wired to the shared preparation and execution functions. We have not completed the public-ingress investigation: which callers can pass the flag, how the interface presents it, and whether it is formally specified as a shared restriction.

This warrants a narrow contract review. It is not a demonstrated live privilege bypass, and no fix has been implemented. The study did not change product code or authorize a new general-purpose permission framework.

5. Test how restrictions fail, not just whether approval succeeds

For each important restriction, record where it enters, where approval binds it, which executor branches consume it, what rejection looks like, and whether the original object remains unchanged.

Tests should come in pairs: a legitimate case that completes, and a prohibited case that is refused without producing the forbidden effect. Checking only an error string can miss another false protection: modifying the file first and reporting an error afterward.

Test each operation mode separately. A positive write test does not cover copy and move. Shared input types do not establish shared enforcement.

Recording a restriction creates an audit trail. Enforcing it creates the capability a caller can actually rely on.

Evidence and limits

This article uses E0–E3, twice each, from the formal dataset. The companion article uses different rows from the same research batch. The English evidence guide, de-identified observations, and Chinese guide are publicly provided with this article.

All files were synthetic and isolated. There is no production loss-rate sample, live HTTP/panel/agent end-to-end test, or multi-process overwrite-race test. Findings are bounded by the pinned source and Windows environment. Record-integrity checks are not product reruns or independent QA. No repair, merge, or release is claimed.

Original English article · 中文版

Research repository

The Receipt Was Corrupted. Why Didn't the Task Start Again?

joinwell52 — Tue, 08 Sep 2026 07:16:16 +0000

We deliberately corrupted a command's completion record and submitted the same command again. The system did call the function responsible for execution a second time.

But do two function calls necessarily start the task twice? Once we connected the function to the product's actual dispatcher, the answer was no. The simulated execution interface still started only once, leaving one execution-attempt record.

Counting function calls alone could easily lead to a premature conclusion of duplicate task execution.

We call that function the execution callback below. The simulated interface replaces the SDK through which the runtime connects to a model. It simulates a start in memory, without a real model, payment, email or code push.

Another callback is not another start. Another start is not, by itself, evidence of a duplicated external effect.

This is not an account of a production duplicate-dispatch incident. It is how counterexamples made us withdraw a premature engineering conclusion.

1. Why can a receipt affect the next execution?

CodeFlowMu is our local multi-agent collaboration system. FCoP organizes task identity and business records; the runtime handles dispatch, sessions and technical execution.

When a user asks to retry a task, the system must both handle the command and record its processing. A response may be lost or a process restarted. On seeing the same request later, the system must distinguish a new command from one already handled.

An idempotency key identifies the same intent. A command receipt is typically recorded as pending while being processed, then appended as completed when a result exists. Receiving the same key again can return the old result instead of repeating the operation.

We first inspected selected historical files to avoid mistaking event counts for incidents.

Historical sample read September 8	Count
Task-command receipt rows	10
Distinct idempotency keys	5
Pending / completed	5 / 5
Malformed / exactly duplicated rows	0 / 0

The records were from September 5. Ten rows represented state events for five commands, not five duplicate dispatches. They did not contain the corruption later injected by the experiment.

We therefore asked in isolated fixtures: if a completion receipt can no longer be read, how does the system judge the same retry request?

2. First round: the bytes remain, but the readable state moves backward

The current receipt reader searches lines in reverse order. It skips an unparseable line or a record with an unrecognized schema version, continuing to a readable match.

This tolerates unrelated bad lines, but it also means that a damaged completion may reveal an older pending receipt. If both target records are unreadable, the reader may return null: no usable record found.

Neither the inspected code nor the experiment showed the reader deleting original bytes. The difference concerns the evidence available to the caller, not what remains on disk.

To isolate this behavior, we used the real command kernel and receipt store, a fixed governance snapshot permitting retry, and a counting executor. Each retry created a new kernel instance so the previous instance's memory cache could not supply the answer.

Read the table for three patterns first: intact receipts allow replay; damaged target receipts can lead to another call; downstream deduplication can still coalesce the effect. The other controls test the conditions under which those statements hold.

Controlled condition	Second read or handling result	Cumulative callbacks / synthetic effects
Intact completion	Reuses completed result	1 / 1
Unrelated malformed line only	Still reuses target completion	1 / 1
Damaged completion, intact pending	Enters callback again	2 / 2
Both target receipts damaged	No usable receipt; enters callback again	2 / 2
Both damaged, current revision changed	Rejects stale revision	1 / 1
Reader explicitly throws	Store unavailable; no further call	1 / 1
Completion changed to unsupported version	Finds older pending; enters callback again	2 / 2
Damaged completion, intact pending, revision changed	Detects state advance; no further call	1 / 1
Damaged completion, executor deduplicates same key	Callback repeats; effect is coalesced	2 / 1

The explicit reader error was an injected counterexample, not a claim that the current reader reports corruption this way. The unsupported version was also synthetic. The last row's deduplication set lived in the experiment's memory, not a durable database.

The first-round conclusion therefore stops at the call boundary: without an explicit read-anomaly signal, the real kernel can call the executor again; whether the effect repeats depends on that executor. The existing kernel already permits retries from pending when the revision is unchanged, expecting downstream work to coalesce under the same key.

3. Second round: measure whether another execution actually starts

Instead of merely incrementing a counter, the second-round callback entered the product's actual task-starting flow. Components responsible for dispatch, attempt persistence, task lifecycle and sessions all came from the current product implementation.

A fixed governance snapshot permitted retry so the request could reach that layer. The final execution interface was still the in-memory substitute described above. We did not issue a request through the web interface: research code passed the same key and dispatch arguments following the web service's retry logic. This exercised actual dispatch components, not the entire web entrance end to end.

This answers a narrower question closer to execution: does the real dispatch chain turn the repeated call into another attempt and another SDK start?

Every scenario first confirmed that an execution started, then damaged isolated receipts and performed the second step. Counts include that first execution. Look first for two outcomes: original-key scenarios did not start again; the new-key control could start. The intervening rows distinguish cancellation, revision changes and concurrency.

Scenario	Change	Second result	Cumulative SDK starts / attempts
I0	Intact receipts	Receipt-level replay	1 / 1
I1	Completion corrupted	Already dispatched	1 / 1
I2	Both target receipts corrupted	Already dispatched	1 / 1
I3	I2, new process reads original disk state	Already dispatched	1 / 1
I4	I2, cancel original session, retry original key	Already dispatched	1 / 1
I5	I2, governance revision changed	Kernel rejects stale revision	1 / 1
I6	I2, eight kernels concurrently call one dispatcher	All eight return already dispatched	1 / 1
I7	Cancel, then use a new command key	New attempt and start	2 / 2

Figure 1. Cumulative counts include the first execution. The protection layers listed are not a claim that I2 traversed every check. Synthetic SDK starts are not external effects. Source: formal I2/I7 observations and boundaries.

Open full-size figure

I1 and I2 correct the most dangerous extrapolation from round one. The receipt could no longer tell the kernel that processing had completed, but session and lease evidence still told the dispatcher not to start again.

I4 adds that cancelling a session does not turn the same command into a new one. Actual reconciliation moved the old attempt to a terminal state. The same key continued to reuse that attempt without another start. Here, the internal phrase “already dispatched” must not be translated as “the old process is still running.”

I7 is an essential control. Without it, readers could suspect the system simply never executes anything. With a different key, the new command allowed by synthetic governance created a second attempt. That is not a same-key duplicate effect, and it does not mean changing a key grants human authority. We did not test the complete authentication and approval interface.

4. The second start was not stopped by one all-powerful receipt

The source explains the observations.

First, the dispatcher checks sessions and active execution leases for the current task and round. A lease describes execution occupancy: who holds which attempt for a task. It is not a business conclusion that the task is complete. Existing sessions or active occupancy should not naturally grant another start merely because dispatch is entered again.

Second, the attempt store reuses records by idempotency key. offer() finds or proposes an attempt; claim() tries to acquire occupancy. These are not equivalent to “one call, one start.” Terminal attempts and conflicting leases have their own rejection conditions.

Finally, one dispatcher serializes redispatch for the same task. Preventing duplicate attempt records alone would not prevent two flows from simultaneously changing task placement. I6 therefore tests the single-instance protection around the larger operation.

I3 exposes a detail worth explaining. The new process has no old in-memory running handle, but the session list still reads a durable running record, and the lease remains. This conservatively blocks a second start. It does not establish that the original executor is alive or that work has recovered.

We did not run full startup recovery or simulate hardware power loss. “The new process retained existing occupancy evidence” accurately describes the result. “Power-loss recovery passed” would not.

5. Why external log repair cannot be imported as a local defect

One starting point was Orca's session-log repair proposal. It addresses a different risk: repair deleting log content after a damaged position, while provider-side conversation history cannot reconstruct the application's own receipts.

At the September 8 research snapshot, the proposal was still a draft. It preserves the original log, publishes the readable prefix as a new generation, and restricts writes for unsupported versions. This was not a released capability independently verified by us. Orca #19399

The two cases are not equivalent. Orca's proposal removes destructive repair; the reader in our experiment did not delete original text. The common concern is that recovery needs its own evidence. Reopening a conversation, or having the model remember it, does not establish that recovery's decision evidence stayed unchanged.

Nor do all local stores skip bad input. The inspected single-record session store distinguishes missing files from corrupt ones. Corrupt operation approvals return an explicit error. These readers also preserve the original bytes. Such counterexamples prevent generalizing one JSONL reader into a claim that the whole system treats corruption as absence.

6. The engineering question is read health, not a verdict of duplicate execution

The second round did not remove the read-layer anomaly. A damaged completion can still leave the caller seeing only an older pending receipt or no usable record, without knowing that corruption lies behind that answer.

Existing execution protections held in the tested scenarios, so the review question should be narrower: can the read result express both which record was found and whether reading was healthy? Should absence, corruption and unsupported versions carry distinct diagnostics? Which commands should those diagnostics affect, and which signals should support explanation or reconciliation only?

This does not immediately prescribe stopping the whole system for one bad line. Unrelated damage, a usable target receipt, version compatibility and actual execution occupancy need separate handling. A blanket block could cut off recovery paths that already work.

A reusable research method is to count each layer: original bytes retained, result read, callback entries, attempts created, executions started and external effects produced. Make conclusions only about the layer actually measured.

Reliability research is not about proving a problem as quickly as possible. It is about locating the problem where a counterexample from the next layer can no longer overturn it.

Here, the callback arrived twice without starting the task again. Preserve the protections that worked; clarify the read anomaly that remains unexpressed.

Research and evidence boundaries

The fixed first-party source was c008d9db91a21136fc61a4f60314e22db395d5d2, examined September 8. Round one contained 18 module controls. Round two contained eight dispatch scenarios and four identity probes. Each set had two formal runs. This article uses related subsets, not repetition counts as incident samples or a reliability percentage.

All corruption was injected into isolated fixtures. Kernel, store, dispatch and session components were real; governance inputs and SDK were synthetic. No real external effects were tested. Eight-way concurrency shared one dispatcher in one process. The cross-process case was sequential handoff, not a cross-process write-lock test. Hardware power loss, all commands, all execution platforms and independent QA remain outside scope.

English evidence guide · Chinese guide. Sanitized per-run observations, source hashes, aggregate history and a record checker are public. The checker verifies saved observations, not a fresh execution of the product. Original operational records, complete before/after fixtures and full product replay remain access-restricted. The research changed no product code and authorized no recovery implementation.

Original English article · 中文版

Research repository

Same Agent, Different Person: Whose Permissions Should the Next Action Use?

joinwell52 — Tue, 08 Sep 2026 07:16:00 +0000

A digital employee handles code for Alice, then accepts a new instruction from Bob. The conversation continues, the task is unchanged, and the same agent does the work.

Whose permissions should it use for its next repository operation?

This is an illustrative situation, not an account incident we discovered. It sounds like switching a login, but it involves another timeline: Alice's action might already have started, Bob's instruction has just been accepted, and previously queued work may still belong to Alice.

Always following the latest speaker could reassign an older instruction to the wrong person. Always retaining the credentials from agent startup could let a new instruction continue using the previous person's access.

A session can continue without making responsibility and authority automatically continuous.

1. Paperclip changes operation-time identity, not just chat identity

Paperclip is an open-source application for organizing and managing AI agent work. Its concrete problem is that several people can instruct the same agent on the same task. The task assignee is not necessarily the person responsible for the instruction behind the current action.

A change merged on September 7, 2026 separates task ownership from execution identity and selects credentials when a managed GitHub operation begins. An action that has already started retains its captured identity. A retry without a new instruction must not switch identity merely because someone spoke later. The author reports switching tests involving two users and two GitHub accounts; we did not rerun those account tests. Paperclip #13005

The boundary matters: this concerns credential selection for managed operations. It does not promise isolation from arbitrary code deliberately copying credentials within the same execution principal. Risk statement

What is worth learning first is not the component count, but the separation of four questions:

Question	Identity or evidence needed
Who executes the work?	The agent's execution identity
Which instruction caused this action?	A reference to the accepted instruction
Who is responsible for that instruction?	An authenticated, policy-resolved person
What access may this action actually use?	Credentials and authority checked at the action boundary

These can be connected, but similar names do not make them interchangeable. A developer role describes a responsibility, not which colleague issued an instruction. A message originating from Alice does not automatically establish Alice's authority to request its contents.

2. Does our own system actually have the same problem?

CodeFlowMu is our locally running multi-agent collaboration system. FCoP task, report and review records organize its work. The runtime handles sessions and technical execution without replacing management roles' business decisions.

It would be easy to see an external project's shared-identity feature and announce that we need the same feature. That skips the important question: what does our own evidence establish?

We inspected selected local records on September 8. The records themselves were from September 5, not new incidents that day.

Selected data	Actual unit	What it tells us
Operation approvals	1 approval record	The deciding role was ADMIN; this cannot estimate authenticated-person coverage
Task-command receipts	10 rows for 5 idempotency keys	Each key has processing and completed events; these are not 10 independent tasks
Skill invocations	25 rows and 25 invocation IDs	9 have nonempty session IDs; the remainder must be distinguished from legitimate sessionless work before judging them

No malformed rows appeared in this selection. Several explicitly named human-identity, instruction-responsibility and credential-grant fields were absent, but this describes the selected fields and records only. Other modules may hold equivalent information.

The history therefore does not support a claim that cross-person credential misuse occurred. Its useful message is narrower: records describing operations, commands, roles and invocations cannot simply be read as an authenticated human responsibility chain.

The next question is not where to add a person's name, but whose identity the current request already binds.

3. A different agent does not automatically inherit an old approval

We called the actual operation approval service and saved its real record format. After approval checks pass, the service invokes a function to do the work: the execution callback. In this experiment, that function only counted calls instead of performing an operation. No repository was connected.

A request digest is an operation-identity check: changing protected request contents should prevent an old approval from being used as though the request were unchanged.

Control	Observation	Supported interpretation
Check and execute the original request	1 callback	The basic allowed path works
Change only the subject from one agent to another	Digest changes; `APPROVAL_STALE`; 0 callbacks	An old approval cannot directly be used for another request subject
Change only the session ID	Same digest; 1 callback	This digest excludes session identity; this does not authorize an entire takeover workflow
Reuse a task-command key but change its submitting actor	`idempotency_key_conflict`; no second callback	The command layer also checks actor differences

The second round also called the real operation-policy builder instead of constructing the request entirely in research code. It derived subject.actor from agentId. Changing the agent changed the digest of the actual constructed request.

The protection exists. It answers which agent submitted the operation, not which authenticated human is responsible for it. Those questions need a connection; one field does not inherently answer both.

4. The reproduced gap is much narrower than an incomplete identity system

To decide whose permissions an action should use, we first need to know which instruction caused it. We therefore checked a more basic question: can the original instruction reference be followed all the way into the execution session? This is one part of responsibility tracing, not a test of human authorization.

Following the chat-continuation path revealed a placement difference. The message identifier reached the command, but it was saved when supplied at the outer level and absent from its session field when supplied inside continuation information.

Specifically, the continuation command retained a chat reference and placed the triggering message identifier, trigger_chat_id, in continuation context. The dispatcher passed that nested context to the session manager. When persisting the triggering-message field, however, the session manager read only the top level.

This is a simplified field-location illustration, not an additional experiment:

context.trigger_chat_id
    -> session.runtime_trigger_chat_id is saved

context.continuation.trigger_chat_id
    -> corresponding session field is not saved

We placed the same source field at each location and called the actual session manager with an in-memory execution adapter instead of a real model.

Input placement	Triggering message saved in session?	Logical execution ID also supplied: saved?
Top-level context	Yes	Yes
Nested continuation context	No	Yes

Figure 1. J1/J2. Absence concerns one session field, not the loss of all chat or command provenance. Source: formal first-party observations and boundaries.

Open full-size figure

The entire session record had not failed to save: the logical execution ID supplied in the same call was persisted. The missing item was the triggering message ID. The code matches the observation: it searches both locations for the logical execution ID, but only the top level for the triggering message.

A separate isolated scenario started through the actual dispatcher also lacked the corresponding triggering-message field in its session record. The four identity probes produced consistent results in two formal runs.

The confirmed gap is therefore specific: source information reached the command, but was not projected through the designated session field.

This relates to Paperclip's question without being the same problem. Chat and command evidence still retain references, so history has not disappeared. A message ID is not a human credential, so repairing this projection would not complete multi-person authorization.

Its practical value is that a query depending on this session field might see no source reference without knowing that the previous layer received one. That hop deserves an explanation before building a chain from an operation back to its instruction.

5. Where should the research lead, rather than what should be built immediately?

This research does not call for immediately rebuilding an identity platform. Retain existing agent-subject binding; review the source-reference handoff contract now; establish a separate requirement and authority boundary before adding shared credentials.

The triggering-message projection difference is a narrow engineering-review topic: who supplies it, which entrances use nested context, how conflicting top-level and nested values should be handled, and which queries actually depend on the field. Those need a clear read/write contract, not an identity platform inferred from one missing value.

Shared credentials need an actual product requirement first. If different people must direct the same digital employee over time, further questions include when instructions are accepted, how responsibility is authenticated, when an old action fixes its identity, how a new action selects authority, and what the underlying platform already supplies. This experiment neither implemented nor accepted that complete chain.

A useful check for agent-product teams is to select one real action and trace backward through its credential, authority, responsible person and original instruction. At the first hop that can only answer “probably this person,” clarify the evidence for that hop.

Remembering who said what is not the same as knowing whose permissions each action may use. Session continuity keeps work connected; responsibility continuity keeps actions attributable.

Research and evidence boundaries

The first-party experiments used fixed source commit c008d9db91a21136fc61a4f60314e22db395d5d2 on Windows with Node v24.16.0. Approval tests used counting callbacks; session tests used an in-memory SDK, not real model tools. Task lookup and governance dependencies of the chat-command builder were synthetic. We did not test authentication or credential switching between two human accounts. These are bounded source and probe observations, not platform-wide security assurance or independent QA.

English evidence guide · Chinese guide. The public bundle contains sanitized observations, source hashes, aggregate historical counts and a record checker. It checks saved evidence, not a fresh product execution; original operational records and full product replay remain access-restricted. The historical sample did not contain the hypothetical multi-person credential incident. No product change or development authorization resulted from this research.

Original English article · 中文版

Research repository

MIIT's AI SME Entrepreneurship Support Plan Explained: An AI Startup Ecosystem Is Taking Shape

joinwell52 — Tue, 08 Sep 2026 04:15:43 +0000

On September 4, 2026, the General Office of China's Ministry of Industry and Information Technology published the Artificial Intelligence SME Entrepreneurship Support Plan (2026–2028).

The document is numbered Gong Xin Ting Qi Ye [2026] No. 31. It was dated August 29 and sent to the authorities responsible for small and medium-sized enterprises in every province, autonomous region, municipality, city specifically designated in the state plan, and the Xinjiang Production and Construction Corps, asking them to implement it in light of local conditions.

Reading it merely as another policy that “supports AI companies” would understate its significance.

In seven pages and 15 specific measures, the plan addresses a larger question: as artificial intelligence moves from competition among a small number of model companies into broad adoption across industries, who will complete the last mile?

The answer is becoming clearer. China does not intend to rely only on a few foundation-model companies or large internet platforms. The policy seeks to cultivate AI SMEs, AI-native firms, agent developers, open-source entrepreneurs, AI application service providers, and even “one-person companies” and “super individuals.” Together, they can become the capillary network that takes AI into millions of enterprises and real industrial settings.

That is the central significance of the plan.

1. Start with the policy sequence, not the subsidy question

The meaning of an industrial policy often becomes clear only when it is placed in sequence with the policies around it.

In August 2025, the State Council issued its opinion on deepening the “AI Plus” initiative. It set a target for adoption of next-generation intelligent terminals and agents to exceed 70% by 2027 and 90% by 2030. The question was no longer whether China should develop AI, but how AI would enter science, industry, consumption, public services, and governance at scale.

The 15th Five-Year Plan then placed AI Plus inside a broader national framework for digital and intelligent development. It called for progress in multimodal AI, agents, embodied intelligence, and collective intelligence; encouraged AI-native businesses; proposed national pilot-scale bases for AI applications; and called for stronger digital enablement services for SMEs and a more developed open-source ecosystem.

Beginning in spring 2026, MIIT policy moved visibly into implementation.

In April, a program for inclusive computing sought to make computing resources easier and cheaper for SMEs to access. It referred to the China Computing Network Platform, SME service zones, “computing banks,” “computing supermarkets,” and vouchers for computing, storage, and network transport.

In July, an updated guide for small, fast, lightweight, and precise digital products defined the kinds of tools suitable for SMEs: affordable products that can be deployed quickly and solve specific problems, rather than large systems by default.

On August 31, MIIT published Document No. 414, a special action to cultivate AI application service providers. It set goals for a national provider pool of more than 2,000 by the end of 2026 and no fewer than 3,000 by the end of 2027. That policy asks who will consult, implement, operate, and govern enterprise AI.

On September 3, ten departments jointly issued the SME Development Plan for the 15th Five-Year period, bringing AI entrepreneurship, transformation, management, and services into the five-year SME policy framework. One day later, the AI SME Entrepreneurship Support Plan was published.

Together, the sequence is coherent:

Set national AI Plus goals → build computing infrastructure → cultivate SME-ready AI products → develop application service providers → expand the supply of AI startups → connect that supply with the needs of millions of SMEs.

The new plan is therefore not an isolated policy. It is building the entrepreneurial supply layer for China's AI application economy.

Figure 1. Five policy moves progress from strategy, infrastructure, products, and delivery to the cultivation of startups. Select the image to view it at full size.

2. The important change: policy now discusses one-person companies and super individuals

The most widely repeated sentence in the plan may be this:

Local authorities are encouraged to provide inclusive support for micro-entities such as “one-person companies” and super individuals that use intelligent tools for agile entrepreneurship.

It is immediately followed by another statement:

Accelerate the cultivation of AI-native enterprises and support emerging groups, including core contributors to open-source communities and agent developers, in exploring paths to commercialization.

These statements do not create a new business-registration category called a “one-person company,” nor do they mean that anyone using AI automatically qualifies for a subsidy. Their deeper meaning is that national policy is beginning to recognize that AI changes the smallest viable unit of enterprise formation.

In the past, starting a software business usually required a product manager, developers, designers, marketing, customer service, and operations. Today, one person who can use large models, agents, and automation effectively can theoretically mobilize capabilities that once required a small team.

The key idea is not the literal number of people. It is that one person may now work with a digital production team built from AI.

This is why the plan places one-person companies alongside AI-native firms, core open-source contributors, and agent developers. Its concept of an AI startup is no longer limited to a conventional software firm with an office, dozens of employees, and a complete organizational chart. Very small organizations may possess substantial capacity for knowledge work, software development, and service delivery.

AI-driven organizational productivity has entered national SME entrepreneurship policy in a direct and visible form.

Figure 2. A one-person company's significance lies in expanding what one founder can produce with AI, while goals, judgment, and responsibility remain human.

3. Naming agent developers turns Agent from a feature into an industrial role

The words “agent developers” deserve particular attention.

The State Council's AI Plus policy already identified agents as a major form of intelligent application and set adoption targets for 2027 and 2030. MIIT now includes agent developers in a policy for cultivating entrepreneurial actors.

The first policy answers whether agents will become an important application form. The second begins to answer who will create them.

Together, they suggest that Agent is moving beyond a feature offered by model vendors and into an independent industrial ecosystem. One group supplies models; another supplies computing; others build agent tools and frameworks, create industry agents, deliver them to enterprises, and operate, govern, secure, and audit them after deployment.

This is also why Document No. 414 and the September 4 plan should be read together. Document No. 414 develops a delivery workforce of AI application service providers. The new plan expands the supply of innovative AI ventures. One addresses delivery; the other addresses entrepreneurship and innovation.

China is organizing more than a large-model industry. It is forming a service chain around the application of AI.

4. The policy builds both AI supply and SME demand

An easy-to-miss sentence near the beginning defines the plan's economic structure: it seeks to combine innovation by AI SMEs with the use of AI to support the high-quality development of SMEs more broadly.

On one side are AI SMEs. They need markets, computing resources, data, finance, and real application settings.

On the other side are China's very large number of ordinary SMEs. They need AI to reduce cost, improve productivity, and strengthen R&D, production, sales, and management.

The policy seeks to connect the two.

It calls for a group of small, fast, lightweight, and precise AI solutions for core business fields such as management, manufacturing, and R&D. It also calls for a virtuous cycle between SMEs exploring AI applications and AI SMEs pursuing innovation.

This means that one of the largest AI markets may not be consumers buying one more chatbot subscription. It may be millions of SMEs purchasing modestly priced, narrowly scoped products and digital workers that solve operating problems directly: procurement assistants, sales agents, customer-service agents, R&D assistants, quality analysis, document processing, market intelligence, finance support, knowledge management, and supply-chain analysis.

A large enterprise may procure a digitalization project worth tens of millions of yuan. A small manufacturer will not. It may, however, pay thousands or tens of thousands of yuan per month for a few AI systems that actually work inside its processes. At sufficient scale, this becomes a market very different from traditional enterprise software.

Figure 3. AI ventures provide focused products, while real SME needs drive those products toward maturity.

5. The hardest startup problem is not technology, but the first customer

China does not lack people able to build AI products. The harder question is: once a product exists, who will let the startup test it in a real setting?

In manufacturing, medicine, materials, and other industries, data, processes, and test environments are usually controlled by incumbent organizations. A technically capable startup may still lack real data, a representative environment, industry customers, and any opportunity to prove its product.

The plan therefore devotes considerable attention to application scenarios. Through activities connecting large and small enterprises and campaigns for AI-enabled industrialization, it asks leading enterprises to open settings for experimental validation, demonstration applications, and scaled commercial use. It also proposes selecting “AI entrepreneurship scenario partners” that can help startups identify scenarios, analyze needs, conduct pilots, and promote adoption.

The three stages matter:

Experimental validation → demonstration application → scaled commercial use.

This is more than holding another startup competition. It attempts to create a route into the market.

The plan also says that incubators should move beyond providing inexpensive office space. Computing, industry data, and application scenarios should become core services. For an AI startup, these resources may be more valuable than a low-rent office. The practical questions are whether it can obtain a model, computing, real data, a customer willing to run a test, and eventually a first order.

Entrepreneurship support is being redefined around those constraints.

Figure 4. The plan tries to turn real application settings into a market-entry route: validation, demonstration, and scaled commercial use.

6. Open source is becoming infrastructure for AI entrepreneurship

The plan makes “deepening open-source ecosystem enablement” one of its four major work areas.

It calls for support for a national AI open-source community; contributions of models, tools, and datasets; adoption of open foundation models, development frameworks, toolchains, and datasets; services for model selection, fine-tuning, performance evaluation, and inference deployment; software, vertical models, and industry solutions built on open-source results; and mechanisms for commercializing those results.

The emerging policy judgment is clear: not every AI startup should rebuild a foundation model from scratch.

A more sustainable industrial structure has a small number of companies investing heavily in foundation models and infrastructure, with many more firms using open models, tools, and data to build engineering systems, products, agents, and industry applications. Mature foundation models lower the entry threshold above them. Richer open-source resources let small companies enter the market faster.

Open source is therefore no longer treated only as a way to develop software. It is beginning to act as public infrastructure for an industry.

7. A new signal: open-source stars enter the government's view of company discovery

The final part of the plan contains a detail that technical entrepreneurs should not overlook. It asks SME authorities to use AI and big-data analysis, together with indicators such as leading talent, invention patents, highly cited papers, and the number of stars on open-source projects, to identify high-growth-potential companies proactively.

Two phrases matter.

The first is open-source project stars. Observable influence on communities such as GitHub and AtomGit now appears directly in policy language about identifying growth companies.

The second is proactive identification. The traditional route into a government cultivation program required an enterprise to find a notice, prepare documents, submit an application, and wait for evaluation. The new wording suggests that public service systems may increasingly use publicly available signals to discover promising firms.

Patents, papers, talent, and open-source projects are all signals. This does not mean that a high GitHub star count automatically makes a company a specialized and innovative enterprise, or that stars lead directly to subsidies. It means that open-source influence now has formal visibility in policy.

This matters for software, development-tool, agent, and infrastructure startups that may have low early revenue, few employees, and little fixed capital. Traditional industrial metrics can miss them. Code contribution, project influence, and developer ecosystems may reveal their technical value earlier.

8. The plan gives specific three-year targets

By 2028, the plan aims to:

cultivate more than 10,000 new technology-oriented and innovative SMEs;
bring the number of specialized and innovative “Little Giant” enterprises in AI to more than 2,000;
establish 10 high-standard technology business incubators;
establish 10 national SME public service demonstration platforms or bases; and
cultivate 10 national-level distinctive SME industry clusters.

For context, MIIT has reported that since the beginning of the 14th Five-Year period China has cultivated approximately 17,600 Little Giant firms, more than 140,000 specialized and innovative SMEs, and more than 600,000 technology-oriented and innovative SMEs.

Setting a separate target of more than 2,000 Little Giant firms in AI is therefore not merely a headcount target. It places AI inside the national ladder for cultivating high-quality SMEs.

Nor does the policy expect these companies to remain small startups forever. Its intended progression is explicit:

Entrepreneurial entity → technology-oriented and innovative SME → specialized and innovative SME → Little Giant → gazelle and unicorn.

Figure 5. The three-year quantitative targets sit alongside a growth ladder from startup entities to gazelles and unicorns.

9. Support is moving beyond a single fiscal subsidy

The plan calls for a guiding role from the National SME Development Fund, the National AI Industry Investment Fund, and the National Integrated Circuit Industry Investment Fund. It also seeks to mobilize private capital across seed, startup, and growth stages.

More unusually, it proposes exploring computing contributions as equity, data contributions as equity, and investment-incubation coordination.

AI companies have a different cost structure from many traditional firms. Their largest costs may be tokens, GPUs, data, model invocation, testing environments, and access to industry scenarios rather than land and factories. Policy support must therefore address different productive inputs.

For an early AI venture, RMB 500,000 of computing resources, a high-quality industry dataset, six months in a real operating environment provided by a large enterprise, and seed capital may be more useful than conventional support tied to physical premises. The evolution of policy instruments follows the evolution of the industry itself.

10. Why cultivate AI SMEs when large technology companies already invest heavily?

Foundation models cannot solve every industry's last-mile problem.

China combines a comprehensive industrial system with an enormous number of SMEs. Bringing AI into those organizations requires detailed knowledge: how an automotive-parts firm conducts quality inspection, how a chemical company procures materials, how an export business manages customers, how a machinery factory quotes work, how a logistics firm schedules shifts, or how a materials company reviews R&D literature.

A handful of foundation-model firms cannot solve all of these problems. Thousands of companies must experiment across narrow sectors.

That is the role of SMEs. They do not compete mainly on foundation-model scale. They compete through scenario innovation and business-model innovation.

The intended structure may therefore be:

foundation-model companies provide the base;
computing and data infrastructure provide shared resources;
large numbers of AI startups create products;
AI application service providers complete delivery; and
millions of SMEs become the application market.

That is the industrial structure required for AI Plus to reach broad adoption.

11. For Agent entrepreneurs, the opportunity is unlikely to be another chatbot

From an entrepreneur's perspective, the plan increasingly favors products that can enter real industrial processes, keep working, be deployed and delivered, support continuing operations, and meet governance requirements. It does not reward an AI demo merely for existing.

This explains why the same document discusses agents, SME-ready products, industry applications, service providers, data, computing, security, compliance, intellectual property, scenarios, open source, and financing. Together, they form an AI production system.

For infrastructure such as CodeFlowMu and FCoP, the policy direction also points toward a market question. Once enterprises deploy digital workers, attention moves from “How intelligent is the model?” to “How does this digital worker keep operating?”

Does it have an identity and defined responsibilities? Which tools can it invoke, and who authorizes those invocations? How are errors recovered? How do multiple agents coordinate? How is the process audited? Who accepts the result? How are enterprise data and permissions protected?

Foundation models do not automatically solve these questions. As models mature and agents enter enterprises, the importance of infrastructure for agent operations, coordination, governance, audit, and security is likely to rise.

This remains an analysis of policy direction and industrial structure. It does not mean that any particular project has been included in a national support program. Application conditions, evaluation standards, and local implementation still depend on subsequent formal rules.

12. The main development is an industry being organized, not a single subsidy

The easiest question to ask is how much money the state will provide each AI startup. The plan does not answer that question. It is not a specific subsidy application notice and provides no uniform grant amount.

Its larger purpose is to organize an AI entrepreneurship ecosystem: where computing comes from, how data is opened, who supplies real scenarios, which new entrepreneurial actors are recognized, how agent developers and open-source contributors commercialize their work, who delivers products, how capital participates, how compliance is handled, and how public services discover promising companies that are not yet well known.

The plan describes its objective as a “tropical rainforest” ecosystem for entrepreneurship and innovation. A rainforest is not a single giant tree. It contains large and small organisms occupying different layers and supporting one another.

China already has large model companies, internet platforms, telecommunications operators, and computing infrastructure. The next wave may consist of small, specialized firms close to industry sites. Some will have only a few people. Some may have one person supported by many agents.

Conclusion: from the large-model race to adoption across millions of enterprises

For several years, AI competition was often judged by model parameters, benchmark scores, financing, and GPU holdings. These remain important, but the direction of national policy is changing.

The State Council established AI Plus in 2025. The 15th Five-Year Plan placed AI on the main path of economic and social digital development. Computing, open source, SME-ready products, application service providers, agent developers, and AI startups are now being organized one by one.

China's AI industry is entering a second stage: from asking who can build the model to asking who can bring AI into every industry.

The scale of that stage will depend not only on a few dozen large model companies, but also on thousands of application service providers, more than ten thousand AI startups, hundreds of thousands of developers, and millions of SMEs that may use their products.

The plan does more than repeat that AI matters. It begins to answer a practical question: how can ordinary entrepreneurs, developers, and SMEs become active participants in the AI economy?

Terms that rarely appeared in national industrial policy—one-person companies, super individuals, AI-native enterprises, agent developers, and open-source project star counts—now appear in an official ministerial plan. That change may matter more than any single subsidy.

Principal sources and interpretation

Industry implications and stage forecasts in this article are analytical judgments based on public policy documents. They are not commitments regarding eligibility or funding. Actual conditions, timing, and local implementation should be determined from subsequent formal notices.

The File Was Written Successfully. Why Check the Artifact Again? From Remote Sandboxes to Local Workspaces

joinwell52 — Mon, 07 Sep 2026 03:49:20 +0000

In this experiment, the file really was written successfully.

We used CodeFlowMu's actual controlled file executor, then started another process to read the file. The SHA-256 content digest—a fingerprint used to compare bytes—exactly matched the post-write digest in the execution receipt.

Then we did something ordinary: renamed the file within the experiment directory while preserving every byte.

The original path no longer contained the file. The historical receipt still said succeeded.

There was no data-loss incident and no contradiction. The receipt correctly recorded the earlier write. The path check correctly answered whether the file could now be retrieved from its original location.

Renaming is only a minimal intervention. The question is not the filesystem truism that a rename changes a path. It is: when an artifact's present availability changes, what judgments can its historical execution receipt still support?

The mistake would be collapsing these two records into one undifferentiated “task success.” When agents work across processes, workspaces, or remote environments, the distinction affects both what the next turn receives and what a human ultimately gets.

1. Did we approve some content, or a particular operation?

First, move the camera from after the write to before it.

CodeFlowMu is the local multi-agent collaboration system we are developing. It organizes work around task files, execution sessions, and evidence. Its controlled file operations do not simply take text and write it. They construct a request, obtain approval, and check the request again before execution.

Two digests in this chain are easy to confuse.

A content digest identifies bytes. An operation digest identifies the particular operation that was approved. Writing identical text in a different workspace or for another task can preserve the content while changing the operation's identity.

To establish what the current system actually binds, we ran four controls through the real request builder at baseline c008d9db91a21136fc61a4f60314e22db395d5d2. Each used new isolated experiment directories across two rounds.

Control	Observation, matching in both rounds	Misconception it rules out
Rebuild the same request with the same workspace and task	Same operation digest	Identical input does not arbitrarily become a different approval object
After approval, change the target from absent to different existing content; rebuild and execute	`APPROVAL_STALE`; no file effect from this executor	Old approval cannot directly overwrite a changed target
Same proposed bytes, different workspace	Same content digest, different operation digest	Identical content does not mean identical execution location
Same workspace and content, different task	Different operation digest	Location and content do not replace task identity

The second row matters particularly. We did not merely observe a digest change and declare protection effective. We attempted execution. The old approval was rejected as stale, and the intervening content was not overwritten.

These controls do not reach equally far. Changing the workspace or task verifies a different operation digest; the changed-target scenario additionally reaches rejection of the old approval during an execution attempt. Distinguishing request identities does not establish acceptance coverage for every cross-workspace or cross-task execution path.

The source explains why: the request incorporates the working directory, task subject, and target snapshot. The execution entry rebuilds it, and the approval service compares the operation digest.

The controls identify an existing foundation to reuse: requests incorporate workspace and task identity, and the tested target change makes the old approval inapplicable.

2. Why does strong write evidence not answer present delivery?

The fifth scenario specifically examined what happens after writing.

The tested executor writes a temporary file, renames it into the target, and returns a post-operation snapshot. We read the file through a separate process and compared its content digest. Then we renamed it within the experiment directory and separately checked the original path and historical receipt.

Observation point	Current file fact	Execution evidence
Actual executor finishes the write	Target exists	Success receipt includes the post-write digest
A new process reads the file	Read-back digest matches the receipt	Success is not merely an intention held in the original process's memory
After a recoverable rename within the experiment directory	Original path absent; preserved file exists	Original execution receipt still says success

Both rounds matched. A new process reading the file demonstrates visibility outside the writing process. It is not a power-loss experiment and does not establish every durability property after power failure.

Figure 1. B4 observes one file at three points, not three independent artifacts. The complete bytes remain at the preserved location; no file was deleted. Source: this study's controlled observations; AI-generated explanatory illustration.

View full-resolution figure

This simple control corrects two opposite mistakes.

The first over-trusts historical success: because the write succeeded then, no one checks the artifact's location at delivery.

The second rewrites success as failure to keep a single status looking consistent: because the original path is empty now, the earlier write must have failed.

Both lose time. One substitutes the past for the present; the other uses the present to rewrite the past.

A more accurate account can retain all of these facts: what bytes were written then; which task and workspace they belonged to; where they can be retrieved now; and who checked when that they meet the delivery requirement.

We did not invoke business acceptance in this study. It would therefore be wrong to report that “acceptance passed missing artifacts.” We tested the responsibilities of execution evidence, not the correctness of the entire acceptance system.

3. Remote sandboxes make the distinction more expensive

On one persistent computer, the distinction may amount to another path check. In a replaceable remote environment, it can determine whether the previous turn's work survives.

Paperclip is an open-source system for organizing agent work and execution state. Daytona supplies remote sandboxes: isolated environments for running code. Paperclip change #12901 described successful changes remaining only in a remote sandbox. Reuse could appear normal, while replacement could restart from stale host data. The change exports and merges artifacts before successful completion, separates workspace recovery from another model turn, and blocks when unexported bytes cannot be recovered. As checked on September 6, 2026, the PR was merged. The author stated that the paid Daytona suite had not been run locally; we did not independently rerun remote tests either.

That external scenario and our local rename are not the same failure. Their connection is a question: between “the executor did it” and “subsequent work can use it from the agreed location,” is there another handoff that needs confirmation?

The local rename cannot validate remote export or merge correctness. The external change cannot establish that our product has the same deployment chain or the same data risk.

We searched the CodeFlowMu runtime and interface-layer source in this study's defined scope and did not find an equivalent Daytona export-and-sync recovery chain. This is a scoped search result, not a global claim about every dependency or user deployment.

The conclusion is therefore not “build a remote workspace system immediately.” First establish whether real usage requires that handoff. An external problem can suggest a direction; it cannot supply product requirements as facts.

4. Long-running work needs artifacts with a location, an owner, and a time

Restoring a task description is not enough for a long-running agent. If the next turn receives stale files, files from another workspace, or outputs with unverifiable ownership, its working basis may have changed even though the conversation continues.

This does not mean every team needs a new artifact platform. Where artifact handoff actually exists, five questions can examine the existing process:

Where is delivery authoritative? Which directory, repository commit, or storage object is the recognized source, rather than a temporary execution copy?
Whose artifact is it? Can it be linked to the task, execution record, and workspace instead of being claimed by filename alone?
Do the bytes match? Are we checking actual content, or only file existence and the model's description?
When was it checked? Evidence from the previous write is not automatically a current handoff check.
Who owns business acceptance? Proving a write does not authorize the executor to declare the content correct or the task complete.

These are recommendations derived from the study, not new CodeFlowMu interfaces or an authorized development task.

If the real deployment uses remote environments, add an environment-replacement control: preserve the output, change execution environments, and check the identity and bytes received by the next turn. On handoff failure, distinguish recovering existing artifacts from executing the original action again. Recovering completed work and potentially producing another external effect should not be hidden behind an ambiguous Retry button.

Conversely, for local workspace operations, precise request binding, post-operation snapshots, and an explicit delivery check may already be sufficient. Research is valuable when it establishes that a direction does not yet warrant development, too.

5. Do not make one “success” answer every question

Two conclusions survive this study together.

One is affirmative: the tested operation digests distinguish workspace, task, and target changes; the changed-target scenario additionally verifies rejection of the old approval. A new process can also verify actual write results.

The other is a boundary: a historical receipt proves what happened in the past; it does not inherently promise that the artifact remains at the same path forever. Current availability and business delivery require their own judgments, not an inference from one successful write.

The next engineering step is to check location, ownership, bytes, and verification time at a real delivery boundary, then decide whether the existing process is sufficient—not assume a new storage project is needed first.

For a long-running agent, “it was completed” is history worth preserving. “It can be delivered now” is another fact worth checking.

Evidence and scope

This article uses five workspace scenarios, two rounds each, for ten controlled observations. The bilingual evidence companion exports B0–B4 together with the related approval observations, source mappings, probes, and integrity checks. Local paths and process IDs are removed without changing outcomes. Record verification is available independently; product-probe execution additionally requires authorized access to the fixed source and its dependencies.

There is no production loss-rate sample, power-loss test, remote sandbox deletion, real remote synchronization, concurrent branch merge, or PM/QA business acceptance in this study. Publication reviewed existing evidence rather than inventing a new experimental round. The renamed artifact was preserved in full, and no user file was deleted.

Language versions and research

Original English article · 中文版 · Research home

Research home