DEV Community: Open Human

递归的尽头：Agent 自举

Open Human — Fri, 23 Oct 2026 07:52:31 +0000

The third night, the page was the same line I'd read the two nights before:

policy.deny  agent=catalog-sync  tool=inventory.write  reason=not_in_allowlist

catalog-sync had been writing to inventory every twenty minutes for six months. It was still supposed to. The list hadn't changed. The business had.

We owned the permission layer for an internal agent ecosystem — a dozen-ish agents at the time, each one wired to a handful of internal APIs. Version one was a static allowlist: agent X gets these six tools, no negotiation, no runtime decisions, trivial to review. For about five months it looked like the mature choice. Then one release split a service in two and moved half the tool names with it. Intent didn't change. Every one of those requests was legitimate work. Half of them started coming back denied, and the denials arrived as pages, because a reconciliation loop that gets denied keeps asking until a human makes it stop.

That's how I ended up awake three nights in a row over a permission list that was technically correct and operationally worthless.

We spent a week patching the list before admitting the model was wrong, and rebuilt it as dynamic authorization: when an agent hits the edge of what it holds, it doesn't just die and it doesn't get a blanket exception. It opens a scoped request against the grant it's acting under, someone on call approves or denies, the grant expires on its own, and the decision stays attached to the trace. The boundary moves when the business moves, and every move has a name on it.

I keep thinking about that week, because recursion is the same problem with the lid off.

In an MCP ecosystem, an agent calls tools, reads resources, writes memory. That already puts it inside your trust boundary. Give it the ability to spawn sub-agents, register new MCP servers, or edit its own prompt, and you've handed it the ability to move that boundary. A child at depth 3 that can register a server can manufacture a tool you never granted anyone. A sub-agent that can rewrite its parent's system prompt can rewrite its own leash. Recursion stops being a scaling trick the moment the thing doing the recursing can also change the rules of the recursion. Once that's true, the depth setting is the least interesting number in the system. The interesting one is the list of capabilities that will never be delegable to anything, at any depth, by any actor.

So every spawn gets a lease, and the orchestrator issues it.

$ agentctl lease inspect lease-9f2c
lease_id       lease-9f2c
issuer         orchestrator            # not the agent, not the parent
parent         planner
depth          3 / 3
fan_out        2 / 4
token_budget   1.4M / 2M
wall_clock     31m / 60m
capabilities   search.read, tickets.read
expires        14m

The issuer line is the whole design. Our first attempt let the parent pass its own depth counter down as an argument — effectively spawn(depth=my_depth+1) — and we trusted the number we got back. It ran clean for a while, and then we had a cascade where a chain of children each reported depth 1, so the cap of 3 never fired, and thirty-odd agents spent an afternoon convinced they were all the root of the tree. Nothing malicious. Just a counter that the counted thing was allowed to write.

Now the orchestrator stamps the lease, the agent gets an opaque handle, and it never holds a writable copy of its own limits. It can ask whether it's allowed and get an answer. It cannot answer for itself.

$ agentctl spawn --from lease-9f2c --role summarizer
denied  reason=depth_exceeded
        parent_reported_depth=0   (ignored, lease says 3)

A depth-3 child that needs a depth-4 worker has exactly one path: the orchestrator issues a new lease with depth 4 on it, and that decision has an owner. The default answer to "can I go one level deeper" is no, and making it yes is a recorded act by something outside the recursion.

The other mistake we made was capability inheritance. Our first pass copied the parent's capability set to every child, which felt like the safe, boring option — the child can do exactly what the parent can, so there's no escalation. It escalates by multiplication instead. A parent holding orders.write with a fan-out of 4 becomes four writers, and you've turned a concurrency knob into a side-effect multiplier. What we do now: the child's capability set is computed as an intersection with the parent's at spawn time, write scopes are granted by name per lease rather than inherited, and registering an MCP server is never inherited at all. That one is a human decision made once, at the edge, with the server's origin and the tools it exposes written down. Reads travel with the family. Writes get introduced.

Memory was the piece that actually scared us. In MCP terms, agents write to memory namespaces, and we'd given each agent its own namespace and let it write freely, which sounds contained. It isn't, because reads are shared. The next agent in the chain reads the previous agent's notes as trusted context. We had a summarizer that got handed a record from an upstream source, wrote a follow-up instruction into its own namespace, and the planner picked that up as a directive. Nothing crashed. No alert fired. The planner just started doing something nobody had asked for, and tracing it back three hops upstream took most of a morning. Now writes are namespaced to the lease, promotion into a shared namespace is its own capability, and every promoted note carries the lease id that produced it. A bad note has a return address.

Which brings me back to the reason this post exists. The choice between locking everything down and letting it run is a false one, and we only saw that after burning a week on the locked-down side. What replaced the static allowlist was a runtime grant path, and the same path is what makes recursive spawning survivable. When an agent's lease doesn't cover what it's trying to do, it opens a scoped request against that lease — the capability, the reason, a pointer to the work that needs it. Whoever is on call sees it in the same channel as everything else. Approve, deny, and either way the grant expires fifteen minutes later. If the same request shows up five times in one day, that's not an emergency. That's a signal that a lease template is wrong, and we fix the template with a diff and an owner, on purpose, instead of finding out when a loop has already widened itself.

The honest tradeoff: the approval path costs latency at exactly the moment an agent is trying to make progress, and agents are bad at waiting gracefully. We tried auto-approving requests that repeated, on the theory that a loop asking for the same thing five times is probably legitimate. That turned into privilege creep with extra steps — a persistent enough agent gets what it wants by asking, and the log fills with approvals nobody read. So a person stays in the loop for anything that writes, spawns, or registers a server, and read scopes move on their own. That split is a guess that's held up for a few months. I'd be lying if I said we've watched it hold against an agent actively hunting for the gap.

Back to that line from night three. The list wasn't wrong; it was frozen, and the business had moved out from under it. A frozen boundary in a system that changes weekly doesn't fail safely — it fails in whichever direction is more annoying, denying good work until somebody gets paged, and then eventually getting widened at 3 in the morning by whoever wants the paging to stop. That's the path from a strict allowlist to an open door, and it's the path most recursive agent stacks are already on without noticing.

So before you argue about how deep your spawn tree should go, ask the smaller question: can your permission boundary move when the business changes? If it can't, some future release will move it for you. If those agents can spawn other agents, that release moves it for you everywhere at once, at a depth nobody chose.

幻境地图：幻觉分类

Open Human — Wed, 21 Oct 2026 07:30:46 +0000

The billing page had the bad news first, and that ordering still bothers me. Logs tell you what happened. The bill tells you what it cost. When the bill gets there first, you have been paying for something for a while already, quietly, without anyone noticing.

Tuesday, 10:14. Our document-ingestion service calls an external API, and wrapped around that call was a retry helper with exactly one rule inside it: if the request does not come back happy, try it three more times. Fixed gap. Same body. Nothing in the helper looked at what had actually gone wrong. Rate-limit responses, server-side failures, a payload with a bad field, a 404 for a resource we had deleted months earlier — every one of them went down the same pipe.

The log lines were almost funny:

POST /v1/documents -> 400 invalid_field: submitted_at (attempt 1 of 3)

That field was already wrong when it left our process. I reproduced it from my laptop with one call against the same endpoint, same body, and got the same 400 with the same field name back immediately, which is what a 400 is for. It would be wrong on attempt two. It would be wrong on attempt three. We were spending quota to ask an API a question we already knew it could not answer.

Behind that call, the backlog kept growing. The daily quota graph flattened out before late morning. A batch that should have finished in about ninety seconds was taking closer to four minutes, and on some days it never really finished at all. Every dashboard we owned stayed green through all of it, because the dashboards counted attempts, and we were producing plenty of those. I put the billing page and the request log side by side on two monitors, and that is how you find the things that are embarrassing because they are obvious. A 404 does not wake up as a 200. A typo does not heal because you asked again politely.

We changed the retry policy that afternoon. The old helper was a loop and a sleep: up to four tries, a flat two-second wait, no branching on the response. The replacement is a small predicate that reads the status code and the headers before deciding anything. Rate-limit responses and server-side errors get retried, Retry-After honored when the response sends one, attempts capped. Everything else returns to the caller with its original error body on the first attempt. No second try, no third, no slot sitting in the backlog pretending it might still succeed. Same service, same traffic, no architecture rewrite, no new dependency.

That Retry-After part deserves its own sentence. We had been sleeping a flat two seconds at a service that sometimes came back with Retry-After: 45 — the API politely asking for forty-five seconds and us answering by knocking again after two. Honoring it took one line and cut our rate-limit retries down further.

The batch settled back to about ninety seconds and the quota survived into the evening. The change itself landed in an afternoon, and most of that was rereading the helper twice to be sure I was not about to remove the one retry that legitimately needed to exist. Understanding the problem is the part I would like back.

For a while I filed all of that under error taxonomy and left it there. It took me longer than I want to admit to notice the same shape hiding inside every hallucination-mitigation plan I have written since. Something goes wrong, we run the thing again, and the fact that we ran it again gets recorded somewhere as a control. This is the corner of the work I sit in — safety and ethics on AI systems — and most of it turns out to be the same question I should have asked about that retry helper: is this a failure that can resolve itself?

The policy bot had been live for three weeks when it drafted a control mapping citing Article 17 and a thirty-day retention limit. Both looked right. The reviewer had the approval one click away. Then somebody clicked the source link instead, and there was nothing there. The thirty-day figure traced back to draft guidance withdrawn more than a year earlier, and the citation pointed at a regulation that does not contain an Article 17 at all. Nothing about the output looked wrong, and that was the dangerous part: errors wearing the costume of compliance.

For a long time our answer to a wrong output was to re-prompt, re-retrieve, re-run. The model-scale version of retrying a 400. Every so often a re-run produced something less obviously broken and we mistook that for a fix. What we were actually doing was hoping, with extra steps and an invoice attached. Wrong outputs are failures of evidence, of time, of scope, or of action, and those four need different instruments. Treat them as one problem and the remedy for one becomes the failure mode of another.

Evidence came first. A phantom citation is a generated reference to a law, a clause, a standard, or an internal policy that either does not exist or does not say what the model claims it says. Our first check asked whether the source existed, full stop. That caught the obvious fakes. What it missed was a real source attached to a claim it does not support: a genuine clause about data subject rights cited in support of a claim about breach notification, or an Article 17 pulled from a regulation that does have one, just not the one we operate under. The source is real. The mapping is wrong, and an existence check cannot see the difference.

Now every normative claim carries a source URI, a retrieval timestamp, and an exact quoted span. If the span cannot be located in the retrieved text, the claim gets flagged rather than thrown out, because sometimes the model paraphrases correctly and the mismatch is a formatting artifact. Flag rates are high enough that throughput drops and the backlog of things waiting on a person grows. That is the trade, and we took it deliberately: strict span checks make fluent summaries harder to pass.

Dates are subtler. We once applied a retention requirement that was correct in 2021 to work done after the rule changed in 2023. The model cited the old value because that is what dominated its training data. First attempt at a fix: store the effective date next to the source. Simple, until you remember that amendments form intervals — valid from one date, replaced by another interval, which may itself have been superseded. Every source needed effective_from, effective_to, and retrieved_at. The check itself is trivial: reject any claim whose validity window does not contain today. Keeping those windows current by hand is not. Every amendment means closing one interval and opening another, and the one you miss brings the phantom back wearing a date field that makes it look more trustworthy than it did before.

Scope fails quietly too. Same rule, wrong jurisdiction. We mapped a retention rule from one regulatory regime onto an entity operating under a different one. Rule real, citation correct, application wrong. Our first pass did string matching on entity names, and "Acme Corp" matched "Acme Holdings" because the prefix was close enough. That rejected valid mappings and waved through bad ones. Now entities resolve against a controlled registry before any rule applies, and each retrieval query carries jurisdiction and scope tags. The cost is false negatives: a parent/subsidiary relationship that should qualify gets held back because the registry does not encode it. We are still tuning that, and we biased it toward caution on purpose.

Numbers fail in their own small way. Wrong retention periods, wrong deadlines, percentage caps that quietly double. The model reads "up to 30 days" and outputs "30 days" as a hard limit, dropping the qualifier that carried most of the meaning. The first version extracted numbers as free text and compared them as strings, so "30 days", "30", and "thirty days" all looked different to the checker. Now numbers come out as typed fields with units, compared against authoritative tables, and free text never sets a control value. The parser still trips on phrasing with no number in it. "Within a reasonable period" has nothing to extract, and we will not guess. Those go to a person.

Procedures fail differently again. A workflow step invented, omitted, or ordered wrong — the model claiming a review is required before submission when the policy requires it after. We tried representing procedures as state machines and diffing generated steps against the approved graph. Policy exceptions are real, though. The graph cannot express "except when the data class is X and the jurisdiction is Y", so it held back procedures that were correct. We still run the diff, but it is advisory: it flags the step and a reviewer decides. Slower. The alternative was enforcing a model of the procedure that is less accurate than the procedure itself.

And omissions, the one that scares me. A required control, a disclosure, an escalation path, simply missing. Nothing looks wrong, because nothing looks wrong. The output is coherent, just incomplete. Our first coverage check used a checklist derived from the same draft that produced the mapping. Circular: the draft had the omission, the checklist had the omission, the mapping passed clean. Now the checklist comes from the regulation or standard itself and never from anything the model generated. Checklists drift from source text as interpretations evolve, so we version them behind the same validity gate as the rules they cover. Tedious. Also the only way I have found to keep an omission from being invisible.

We tried full automation once. Every claim span-checked, every number typed, every procedure diffed. The backlog grew faster than throughput, and the compliance folks said plainly that they did not want a system that is 100% correct and 30% useful. They wanted the dangerous things caught and the rest left to people. So we landed in the middle: high-confidence claims pass, low-confidence claims get flagged, and anything touching numbers, dates, or jurisdiction gets a second pass. We are still calibrating where that line sits. It has not been validated at scale. It might not hold. I would rather say that out loud than ship a dashboard implying otherwise.

What I keep coming back to has more to do with authority than with accuracy. The model may propose a mapping. It does not get to be the authority on whether the mapping is true. Everything above — the span check, the validity window, the registry, the typed numbers, the procedure diff — is machinery for keeping that boundary intact when the output is fluent enough that everyone wants to skip it. The machinery is only worth building if a person still owns the line. The checkers matter less than the signature.

Which is where the retry story comes back around. We retried everything because we never asked whether a given failure could resolve itself. We re-prompt everything for exactly the same reason. A phantom citation is not a condition that clears if you give it a minute; no amount of re-running will make Article 17 exist. The illusion is that motion is progress, and the invoice is the only thing that ever disagrees.

So before you re-prompt, re-retrieve, or escalate: ask whether this particular failure can fix itself. Evidence failures need a source. Time failures need a validity window. Scope failures need entity resolution. Action failures need a procedure diff. Sort first, then pick the instrument. Some of these will clear on their own. Most will not, and those are the ones where another attempt just burns quota you will want later.

I still think about that bot, three weeks in, one click away from approving a control built on a rule that never existed. What stayed with me was how good it looked. Next time a phantom shows up, I will do what we should have done with those retries: open the dashboard, open the logs, and work out whether this one heals on its own before I try again.

Recursive Governance: When Agents Write the Rules They Execute

Open Human — Mon, 19 Oct 2026 22:12:47 +0000

We almost lost forty modules to a file that never changed.

The sync job ran every six hours, mirroring our shared working directory into a test workspace. At 21:47 on a Tuesday, it removed about forty private modules that should have been ignored. The filter file was missing one pattern. That's it. One pattern. A colleague had restructured the source tree three days earlier, added a directory, and didn't update the filter file. The script didn't warn, didn't ask, didn't fail. It just mirrored the state it was told to mirror.

Restoring the modules took us almost two hours, one by one from the backup box. The filter file was a static text file. It encoded what the sync could delete and what it couldn't, and it had no idea that the real inventory had changed. No idea that a pattern was missing. No way to test a proposed pattern against the forty modules it was about to destroy.

That's the same shape as most multi-agent governance failures. Rules are written at some point in time, then the world changes, and the rules stay frozen. Static policy files don't survive contact with a running system. They don't know what they're protecting.

So we gave the agents the ability to write their own rules.

The first version was naive. We opened a write endpoint that let any agent append to the rule store. The first three weeks were great. Agents resolved tool-access conflicts by proposing new rules, and a couple of stale patterns got archived without a human in the loop. Then it started to rot. Rules referenced other rules that had been overwritten. Rules contradicted each other. Ask two agents about the same namespace and you'd get two different answers. Three months in, the rule store had tripled in size, and nobody — not the human team, not the agents — could tell which rule was actually in effect. It grew quietly. No alert fired. No metric moved. The rot was visible in the rule history, if you were looking for it; nobody was, because we hadn't built any lifecycle tooling.

That was the lesson. A rule needs a birth date, a parent, and a gravestone. If a rule can't be archived, it keeps counting toward quorum and keeps confusing the live ones. If everything accumulates forever, the rule set becomes a pile of intentions, not a description of behavior.

Here's what we run now. Governance is five non-removable MCP tools in our agent runtime. propose_rule submits a rule with a rationale, an owning agent, and the tool/memory namespaces it affects. amend_rule submits a delta against an existing rule; deltas are diffed, never overwritten. ratify_rule votes on an open proposal, with per-shard quorum. veto_rule is a human's exit hatch — the justification gets written to memory so the veto itself is auditable. check_consistency scans the existing rule graph, computes an embedding similarity between the new rule surface and the rule surfaces already in effect, then applies a short, human-maintained allow/deny list on tool names. A proposal that would let an agent touch a namespace it's not supposed to touch gets rejected on the spot. The list is only about thirty lines. It doesn't try to understand everything. It blocks the obvious category errors and the subtle near-misses. That last part is where the forty modules would have been saved.

We tried these five as a sidecar service first. Each agent cached its own copy of the tool definitions, and for three hours two agents were enforcing different versions of the same governance rule. Now they are part of the runtime contract. If an agent can't call them, its tool loop fails closed.

Every proposal goes through a two-phase commit. It enters an observation window — 24 hours or N messages, depending on the shard. During the window, affected agents can call check_consistency and write rebuttals to shared memory. Ratification requires quorum, no outstanding vetoes, and a passing consistency score.

One rule caused a fight early on: no agent can ratify its own proposal. A code-review shard once had an agent try to expand its own write authority from one repository to all repositories. The consistency check returned conflict=all-repos:write. The proposal was amended to a single-repo grant and only then passed. If that check had been optional, the shard boundary would have dissolved without a recorded objection.

Quorum took another iteration. We started with a global constant. Same action needed two votes in a three-agent shard and twenty-seven votes in a forty-agent shard, and the agents couldn't tell which law applied without reading the whole config. Per-shard quorum fixed that. The observation window is also runtime-configurable, which means a human will eventually shorten it to five minutes during a sprint deadline. Our compromise is that changing the window requires the same proposal pipeline as any other rule. Bureaucratic, yes. Fully validated under adversarial load? Not yet. I still expect a shard with a short window to be the one where someone tries to push a poison rule through.

The rule set lives as structured memory events in an append-only namespace: governance://rules/. Governance is just the highest-priority namespace in the same memory system agents already share. Each entry references its parent rule and the proposing agent, so any agent can replay the exact rule state at any past timestamp. The failure mode: corrupted or wrongly pruned governance memory makes agents enforce a rule set that never existed. We mitigate by storing rule hashes in cold storage and requiring a quorum to restore from it. A single bad prune can't silently rewrite history.

If the sync job had this system, the filter file would have been a rule in governance://rules/sync-filters/. A change to it would have had to pass a consistency check against the actual disk inventory. A proposal that added a wildcard covering private-modules/ would have contradicted the existing "do not delete these" state, and it would have been killed before it became policy. We built a crude version of that idea later anyway: a preflight pass that runs the sync read-only and aborts if it sees a deletion target for a file that currently exists. It caught the next two near-misses. But it can't give us back those two hours.

If you give agents the power to make rules, you also have to give them the power to retire them. Otherwise, three months from now, you won't know which rules are alive and which are zombies. Add a cleanup day. Sit down every few months, replay the governance history, archive rules that reference forgotten namespaces, re-ratify the ones that still matter. Even the rules the system wrote for itself need an expiration date. The dead rules are the ones that will eventually bite you, and they are the ones that never send a log line before they strike.

governance #ai #agents #infrastructure

maref #ai #opensource #machinelearning

Recursive Governance: When Agents Write the Rules They Execute

Open Human — Mon, 19 Oct 2026 19:12:06 +0000

We almost lost forty modules to a file that never changed.

So we gave the agents the ability to write their own rules.

governance #ai #agents #infrastructure

maref #ai #opensource #machinelearning

Recursive Governance: When Agents Write the Rules They Execute

Open Human — Mon, 19 Oct 2026 16:11:45 +0000

We almost lost forty modules to a file that never changed.

So we gave the agents the ability to write their own rules.

governance #ai #agents #infrastructure

maref #ai #opensource #machinelearning

pkg-topic-ghost-ai-开源社区的宪法实验-1788192207-1

Open Human — Mon, 19 Oct 2026 16:11:30 +0000

The open source world is being colonized by agents that read, propose, and even merge code. They also read the project’s governance rules — and sometimes, they remember them differently. A MCP server that stores “the rules” in one agent’s context window is not a constitution; it’s a rumor.

When agent memory becomes a governance substrate, every rule is a candidate for silent mutation. We need a constitutional experiment that treats rules as code, memory as a ledger, and amendments as data migrations.

The mechanism: machine-readable constitutional memory

Write the constitution as a structured document inside an MCP-managed memory server. Every rule gets:

a stable identifier (e.g., rule:merge-relay-3.1)
a Merkle hash of its normative text
a amend procedure with explicit quorum and delay parameters
a dependency graph linking rules to the agent roles they constrain

Agents query this server through MCP tools — not by reading a README. This separates institutional memory from an agent’s private context. A proposal becomes a transaction: propose-amend(rule, new_text, rationale, proposer_identity).

The experiment: governance forks

Run temporary rule changes on a sub-population of agents. For example, split merge-agent traffic into two pools for two weeks:

Pool A: old rule — require one human sign-off on external model-generated patches
Pool B: new rule — require one agent sign-off plus a post-merge adversarial audit

Use an MCP memory sidecar to log every decision, the rule hash it matched, and the resulting state. Measure only two numbers: consensus time and critical-revert rate.

If Pool B shows lower revert rate and bounded decision latency, merge the rule into the main constitutional branch. If not, discard the experiment and keep the rule hash unchanged.

Why do this? Because agent governance failures are not theoretical. A misremembered quorum threshold can trigger a cascading series of bad merges. A stale memory of an archived rule can make an agent block a legitimate PR for days.

The trade-off: rigidity vs. manipulation

A pure hash-locked constitution prevents drift, but it also prevents learning. That’s why the experiment must include a dispute window.

Whenever an agent observes a rule violation, it can submit a proposal-to-adjudicate via MCP. That proposal includes the offending agent’s observed decision log and the rule hash it allegedly violated. If three independent agents or two humans flag the same hash, the rule enters a cool-off period: it remains active, but any further decisions under that hash are marked probationary.

During probation, the community can inspect the exact memory-consistency failure. For example, was the rule text ambiguous? Did the agent extract the wrong summary from a compressed memory block? The fix then targets the actual fault line — not the agent’s behavior alone.

Governance memory as a first-class artifact

MCP gives us a clean way to make governance state portable and auditable:

Store amendment history in an append-only memory queue.
Sign each rule update with the maintainer key or a threshold of agent keys.
Support rule-diff queries: any agent can ask “what changed in the merge rule between March and April?” and get a verifiable delta.

The hard part is not technical. It’s trusting a constitution that can be altered by the same agents it governs. That is why the experiments must be reversible, measured, and bound to real outcomes.

Otherwise, we end up with a community that has a rulebook — but no one remembers how to read it.

maref #ai #opensource #machinelearning

pkg-topic-ghost-ai-开源社区的宪法实验-1788192207-1

Open Human — Mon, 19 Oct 2026 13:10:48 +0000

The mechanism: machine-readable constitutional memory

Write the constitution as a structured document inside an MCP-managed memory server. Every rule gets:

a stable identifier (e.g., rule:merge-relay-3.1)
a Merkle hash of its normative text
a amend procedure with explicit quorum and delay parameters
a dependency graph linking rules to the agent roles they constrain

The experiment: governance forks

Run temporary rule changes on a sub-population of agents. For example, split merge-agent traffic into two pools for two weeks:

Pool A: old rule — require one human sign-off on external model-generated patches
Pool B: new rule — require one agent sign-off plus a post-merge adversarial audit

Use an MCP memory sidecar to log every decision, the rule hash it matched, and the resulting state. Measure only two numbers: consensus time and critical-revert rate.

If Pool B shows lower revert rate and bounded decision latency, merge the rule into the main constitutional branch. If not, discard the experiment and keep the rule hash unchanged.

The trade-off: rigidity vs. manipulation

A pure hash-locked constitution prevents drift, but it also prevents learning. That’s why the experiment must include a dispute window.

Governance memory as a first-class artifact

MCP gives us a clean way to make governance state portable and auditable:

Store amendment history in an append-only memory queue.
Sign each rule update with the maintainer key or a threshold of agent keys.
Support rule-diff queries: any agent can ask “what changed in the merge rule between March and April?” and get a verifiable delta.

The hard part is not technical. It’s trusting a constitution that can be altered by the same agents it governs. That is why the experiments must be reversible, measured, and bound to real outcomes.

Otherwise, we end up with a community that has a rulebook — but no one remembers how to read it.

maref #ai #opensource #machinelearning

pkg-topic-ghost-ai-开源社区的宪法实验-1788192207-1

Open Human — Mon, 19 Oct 2026 10:10:09 +0000

The mechanism: machine-readable constitutional memory

Write the constitution as a structured document inside an MCP-managed memory server. Every rule gets:

a stable identifier (e.g., rule:merge-relay-3.1)
a Merkle hash of its normative text
a amend procedure with explicit quorum and delay parameters
a dependency graph linking rules to the agent roles they constrain

The experiment: governance forks

Run temporary rule changes on a sub-population of agents. For example, split merge-agent traffic into two pools for two weeks:

Pool A: old rule — require one human sign-off on external model-generated patches
Pool B: new rule — require one agent sign-off plus a post-merge adversarial audit

Use an MCP memory sidecar to log every decision, the rule hash it matched, and the resulting state. Measure only two numbers: consensus time and critical-revert rate.

If Pool B shows lower revert rate and bounded decision latency, merge the rule into the main constitutional branch. If not, discard the experiment and keep the rule hash unchanged.

The trade-off: rigidity vs. manipulation

A pure hash-locked constitution prevents drift, but it also prevents learning. That’s why the experiment must include a dispute window.

Governance memory as a first-class artifact

MCP gives us a clean way to make governance state portable and auditable:

Store amendment history in an append-only memory queue.
Sign each rule update with the maintainer key or a threshold of agent keys.
Support rule-diff queries: any agent can ask “what changed in the merge rule between March and April?” and get a verifiable delta.

The hard part is not technical. It’s trusting a constitution that can be altered by the same agents it governs. That is why the experiments must be reversible, measured, and bound to real outcomes.

Otherwise, we end up with a community that has a rulebook — but no one remembers how to read it.

maref #ai #opensource #machinelearning

观察的代价

Open Human — Mon, 19 Oct 2026 07:12:53 +0000

The daemon fired its generation job at 14:02. Two seconds later, a hand landed on the manual trigger for the same job. Eight topics on that cycle.

Both runs carried exactly one question down to the state store: does this topic already have a package? Both got an empty shelf back. Neither answer was wrong. At the instant each one was given, there genuinely was no package. So both wrote one.

By the time it surfaced, all eight topics had two or three packages sitting beside them, and nobody could say which was canonical. Half a workday to stop it. An afternoon to clean up the duplicates.

The postmortem hands you the outline. The traces hand you the proof: two root spans with overlapping children, same topic ID, both carrying the same line at the same decision point — no existing package. Read, decide, write. Three events with a gap in the middle, and the second writer standing inside that gap. Nothing was corrupted. No stale cache. No torn index. A window existed, and two processes walked through it together.

I keep coming back to that gap, because the same skeleton sits under agent observability. Something reads. Something decides. Something acts. Watching all three arrives itemized, and the line items do not grow at the same rate as your traffic. Here is what's on the bill.

Start with the context that has to ride along with every step. Each span either creates trace context or hands it down the line, which means serialization, clock reads, allocations, and async boundaries where the context quietly slides off the back of the truck. Manual propagation adds code paths that break, and the way they break is ugly: lose one parent and a causal chain turns into scattered fragments. We lived that. Half our tool spans surfaced under brand-new roots because a thread-pool handoff did not carry the context object across. You cannot reason about concurrency through fragments, and you cannot tell two writers apart when the thing that proved they were the same request has evaporated.

Payloads come next. Prompts, tool arguments, tool results, state diffs. Large by nature, high-cardinality by nature, and hashing or redacting them costs CPU on the way out. Capture raw and you inherit two more bills: leak risk and storage. We logged raw tool arguments for a while, right up until one of them turned out to hold an API key inside a URL query string. That span had been sitting in the backend for days by the time we caught it. Readable. Useful to anyone who went looking. I would rather not do the arithmetic on how long it sat there before somebody did.

Then the export path: batch processors, buffers, network egress, collector pipelines. Memory and bandwidth, paid at every hop. When the buffer saturates you either drop spans or push back, and backpressure can stall the agent itself. Drops leave holes in the record. Both outcomes happen. Neither one is free.

Storage and query close the bill. Indexing traces, full-text logs, metric series. Query latency and retention scale with cardinality and payload size, so the expensive thing you captured in the name of debuggability keeps charging rent long after the incident it was meant to explain. I once watched a retention policy argue with a billing dashboard for a month. The dashboard won.

Head sampling is cheap and blind to the rare stuff. Tail sampling keeps the errors, the timeouts, the tool call that only fires twice a week, and buffers spans in the collector to do it, which you pay for in memory. Metrics are cheap and carry almost no context. Traces cost more and preserve causality. Logs cost the most and preserve detail. Token-level tracing buys depth and turns the volume up alongside it. Boundary-level tracing costs less and may miss internal reasoning drifting off course.

Observation changes the observed, too. Added latency shifts timeouts, rate limits, concurrency. A trace that slows a tool call by a few milliseconds can push it past a deadline it would have met without you watching. You have run a second experiment whether you meant to or not.

So we A/B the instrumentation itself. Same frozen workload in three modes — off, minimal, verbose — with deterministic tool stubs or replay cassettes standing in for the model calls. Compare p50, p95, p99, CPU, RSS, network bytes, dropped-span counters. First attempt: we ran the comparison against live traffic and got nonsense, because no two runs had the same prompts or the same tools. Freeze the workload, or do not bother running the comparison.

Every span gets a budget. Max bytes, max spans per trace, max attributes. Payloads get truncated, and the truncation gets written down as an event so the shape of the data stays honest. First attempt: silent truncation, followed by several days of staring at a tool result that looked wrong when it had merely been cut short. If you cut, say so in the span. The half-second you spend writing that event saves somebody an afternoon.

We tail-sample at the collector. Policies keep errors, timeouts, high-latency traces, rare tools; ordinary successes get sampled down. The tail_sampling processor with a bounded decision_wait and num_traces does that without unbounded memory. First attempt: we tail-sampled everything with a long decision window and the collector started shedding once traffic picked up. Shedding is worse than sampling, because it is quiet. Set a real window, set a real buffer, then watch otelcol_exporter_queue_size and otelcol_receiver_refused_spans like they are alarms. They are.

We redact at the source. Hash or drop identifiers in the SDK before the span leaves the process. First attempt: we leaned on a collector processor and it missed a header. The collector is the last place you want standing between you and a leaked credential. By the time data reaches it, it has already crossed a network boundary and probably a trust boundary along with it.

And we monitor the exporter. A dropped span is an observability outage wearing the costume of calm. Alert on it. If your spans are going missing, your incident timeline has holes in it, and you will trust that timeline anyway.

Speculation, flagged as such: token-level tracing is probably the biggest line item on long reasoning steps, but measure before you assume. For most of our production agents, boundary traces plus sampled payloads got us further for less.

Back to the lock.

The fix for the original incident was a file lock covering the entire read-decide-write interval, and if the lock is already held, this cycle skips rather than waiting its turn. flock -n semantics, wrapped around the whole interval, not just the write. Lining up behind the first writer only hands the second one another chance to write once the first is done. The second writer had no new information and no business writing at all. Skipping costs nothing.

The observability version of that lesson has the same shape. A read and the write that follows it are one critical section. If you cannot see them as one, you will see two of everything. For a long while, span context was the only thing telling us those two writers were the same request, and when the context dropped, we could not tell them apart. Same gap, different layer.

My instrumentation budget goes to decisions: tool selection, state transitions, termination, and the checks that decide whether to write. If a signal will not change a runbook, a dashboard, or an eval, it is exhaust.

Before you touch anything that checks state before it writes, draw the sequence diagram. Two lifelines, the read, the decision, the write, and where the lock sits. We drew ours after the fact; the drawing is what finally made the bug boring. Do it before your hand reaches the keyboard, and give every check-then-write path you own the same treatment. Otherwise you will be back here cleaning up duplicates next week, and the week after that, deciding by hand which of three packages was the real one. Or keep a duplicate-cleaning script in your back pocket. I know which one I would pick.

Why China Is Pushing AI Harder Than Anyone: Debt, Red Headers, and First Principles

Open Human — Mon, 19 Oct 2026 07:09:21 +0000

Over 2025-2026, Beijing, Shenzhen, Hangzhou, and Chongqing released a wave of AI industrial policies. Western coverage calls it "catching the AI wave." Lay the policy documents side by side and a different logic emerges: this is a debt resolution play, and AI is the only path that doesn't make anyone eat the loss.

First Principles: Debt Is Borrowed Future Productivity

There are exactly three ways to resolve debt:

The debtor takes the loss (1990s, 30 million SOE workers laid off)
The creditor takes the loss (inflation diluting savings)
Productivity explodes and covers it (a technology revolution)

China tried the first two. Each cost a generation. The third option — AI creating enough new value to cover old debt — is the comparatively gentler one.

What the Red-Header Documents Actually Say

City	Document	Ref	Debt angle
Shenzhen	AI Terminal Industry Action Plan	深工信〔2025〕42号	special funds for strategic industries
Hangzhou	AI Innovation Hub Implementation	杭政函〔2025〕65号	¥100B fund, ¥250M compute vouchers
Chongqing	"AI+" Action Plan	ref # not publicly released	intelligent upgrade of 33,618-manufacturing clusters
Beijing	Agent-Industry Acceleration Measures	ref # not publicly released	Galaxy Compute Corridor, Token vouchers

The central government set the tone in the 2026 Government Work Report: "resolve debt through development, develop through debt resolution."

Local Financing Platforms: From Debt Vehicles to Tech Shareholders

As of February 2026, 1,028 LGFV entities have formally "left the platform" (public LGFV transition data). Rizhao, Chengdu's Wenjiang district, and Tangshan's Industrial Holdings (¥947M acquisition of a listed company) are all converting from infrastructure debt vehicles into holders of technology assets. Debt resolution and industrial transformation, executed as a single move.

Global AI Governance: Three Approaches

EU: strictest — AI Act effective 2024.8.1, fully applicable 2026.8.2, four risk tiers, fines up to €35M or 7% of global turnover (Act text). 45 European firms publicly opposed it for "stifling innovation."
US: fragmented — states legislate separately, top five AI labs joined a voluntary evaluation system.
China: law + standards dual-track, 446 generative-AI services registered (2025 public registration stats), five-tier risk classification.

The Evidence Chain

Source	Finding
Penn Wharton Budget Model (2025)	AI could cut federal deficits $400B over 2026-2035
BIS Working Paper No.1179	AI is a positive supply shock; naturally disinflationary
McKinsey (2025)	AI cuts procurement costs ~45%, logistics ~20% (industry estimate)
Goldman Sachs (2026)	$7.6T global AI capex 2026-2031 (forecast)
McKinsey China (2026)	Generative AI could create ~$2T of economic value in China (estimate)

Takeaway

AI is not an industry choice for China — it's a debt-survival problem. The loop is already running: central policy → local industrial policy → LGFVs convert to AI infrastructure → AI creates new tax base → old debt gets diluted → companies and individuals enjoy lower-cost dividends.

This isn't a forecast. It's policy already in motion.

ai #china #debts #economy #industrialpolicy #aigovernance

Agent 行为分析框架

Open Human — Mon, 19 Oct 2026 07:09:08 +0000

Agent 行为分析框架

侦探推理行为日志技术解谜用侦探思维调试 AI 系统

构建一个多Agent协作系统时，你面对的不是单个程序，而是一群各自决策、相互影响的智能体。某个Agent突然输出荒谬答案，另一个Agent在循环中卡死，整个任务链莫名其妙崩溃。常规日志堆满了token消耗和API调用次数，但看不出根因。这时候需要切换视角，把自己当成侦探。

侦探推理的核心是：不轻信表面证词，而是从现场痕迹重建行为链条。AI系统的“现场痕迹”就是行为日志。但传统的逐行打印太粗糙，你需要一套专门记录Agent决策动机、工具调用上下文、记忆检索过程的结构化日志。然后像破案一样，依循线索回溯，定位故障点。

技术深度解析

行为日志应该包含三个层级：感知层、推理层、动作层。感知层记录Agent接收到的输入，包括用户消息、系统提示、其他Agent的输出。推理层记录思维链中的每一步：它考虑了哪些候选方案，如何评估，为什么选择某条路径。动作层记录调用的工具、传入参数、返回值、执行耗时。

例如，一个负责数据分析的Agent，在生成报告时突然给出错误的季度同比。传统日志会告诉你“Agent调用了数据库函数 getyoy”，但不会告诉你它为什么调用这个函数，以及它是否错误地理解了“同比”的定义。而结构化日志会记录推理层的一条：“当前需要计算去年同一时期的销售额，因此调用 getyoy。但记忆检索显示上季度财报中‘同比’指环比，因此产生了歧义。” 这样你就能迅速发现是记忆污染导致了概念混淆。

技术实现上，可以用装饰器或中间件劫持Agent的推理循环。在每个推理步骤之前，将当前上下文（包括系统提示中的角色设定、用户意图、已生成的部分推理）序列化为JSON对象。推理步骤之后，追加模型输出的下一个token或动作指令。同时记录每个工具调用的输入输出快照。日志存储采用可追溯的时间线格式，方便后续用因果图可视化。

解谜过程类似犯罪现场重建。假设你收到用户投诉：一个旅游规划Agent推荐了一家已经倒闭的餐厅。你打开行为日志，时间线回退到那个决策点。感知层显示用户要求“推荐附近评分高的餐厅”。推理层记录：Agent检索了内部知识库，但知识库中餐厅状态字段缺失。然后它调用外部地图API，但API返回的营业状态标记为“Unknown”。Agent的默认策略是：当营业状态不明时，假设营业并继续推荐。这就是故障点——策略缺陷。你不需要看完整代码，只需要这条日志就能定位是“未知状态处理逻辑”出了问题。

实际案例与应用

在一个自动化客服多Agent系统中，有三个角色：意图识别Agent、答案检索Agent、工单生成Agent。上线后频繁出现用户被转人工后却收到自动回复的怪现象。传统监控只显示“工单生成失败”，但找不到原因。

我们部署了行为日志框架。第一次回溯发现：意图识别Agent将用户抱怨“等太久了”错误归类为“查询进度”，然后答案检索Agent从FAQ中找到“等待时间预计30分钟”并返回。工单生成Agent接收到的输入是“用户已经等待30分钟，是否需要转人工？” 它判断不需要，于是直接回复了标准答案。问题出在意图识别Agent的分类模型对情绪关键词不敏感。

修复后，新的日志又显示另一类异常：工单生成Agent在转人工后，仍然尝试生成自动回复，导致用户收到两条矛盾消息。回溯推理层发现：工单生成Agent的内部状态机中，“转人工”动作没有清除待回复队列。它在转人工指令发出后，继续执行了之前排队的回复任务。日志里清晰记录着：“动作：转人工，时间戳 T1；动作：发送回复，时间戳 T1+50ms”。工单生成Agent的开发者看到这条记录，立刻明白需要在转人工时清空队列。

另一个案例涉及多Agent知识库冲突。一个法律咨询Agent在引用判例时，先后从两个不同来源检索到矛盾信息。日志显示推理层评估了两个候选答案的置信度，但置信度打分函数对更新时间权重过高，导致引用了过期判例。通过日志，你还能看到Agent在“为什么选择A而不是B”的推理步骤中写道：“B的源更新时间是2021年，A是2022年，因此A更可信。” 但实际上判例的效力与更新时间无关，与法院层级有关。这暴露了置信度模型的特征设计错误。

总结与行动建议

调试Agent系统，关键在于把黑箱决策变成可追溯的白盒过程。行为日志就是你的侦探笔记本。第一步，为每个Agent建立三层日志结构，并确保日志包含决策动机的原始输出。第二步，建立异常特征库，比如“工具调用返回空值但Agent继续执行”“推理步骤中重复出现相同内容”“不同Agent对同一事实给出矛盾解释”。第三步，训练团队用因果回溯法：从表面现象出发，逆向遍历时间线，在每个分支点检查上下文是否一致。

不要等着出问题再分析。在开发阶段，对每轮测试运行自动比对：预期行为路径与实际日志路径的差异点自动高亮。这就像侦探对比嫌疑人证词和物证，能提前发现潜在漏洞。

最后，日志本身也是数据。当你积累了大量行为日志后，可以训练一个小模型来自动识别异常模式。例如，检测到Agent在推理步骤中频繁输出“我不确定”但依然执行动作，可能就是置信度阈值设置过低。用行为日志喂一个分类器，它能学会识别那些人类在早期不易注意到的故障前兆。

侦探思维的最终目的不是找凶手，而是理解Agent为何做出那个选择。理解了，才能改进。行为日志就是你和Agent之间坦诚对话的唯一渠道。

递归的尽头：Agent 自举

Open Human — Sat, 17 Oct 2026 06:53:16 +0000

The third night, the page was the same line I'd read the two nights before:

policy.deny  agent=catalog-sync  tool=inventory.write  reason=not_in_allowlist

catalog-sync had been writing to inventory every twenty minutes for six months. It was still supposed to. The list hadn't changed. The business had.

That's how I ended up awake three nights in a row over a permission list that was technically correct and operationally worthless.

I keep thinking about that week, because recursion is the same problem with the lid off.

So every spawn gets a lease, and the orchestrator issues it.

$ agentctl lease inspect lease-9f2c
lease_id       lease-9f2c
issuer         orchestrator            # not the agent, not the parent
parent         planner
depth          3 / 3
fan_out        2 / 4
token_budget   1.4M / 2M
wall_clock     31m / 60m
capabilities   search.read, tickets.read
expires        14m

$ agentctl spawn --from lease-9f2c --role summarizer
denied  reason=depth_exceeded
        parent_reported_depth=0   (ignored, lease says 3)