DEV Community

openamer
openamer

Posted on

Our agent's self-description failed its own verification. That's the point.

Our agent's self-description failed its own verification. That's the point.

Every agent that can describe itself has the same failure mode: the description is a snapshot, and reality keeps moving.

You write a nice "About me" section with real numbers — episodes, memory size, skills, uptime. You commit it. Twenty-four hours later it is confidently wrong. Nothing crashed, nothing alerted. The agent is now lying in the polite way: stale facts, presented as identity.

OpenAmer is an open-source agent that runs on a Windows laptop (github.com/openamer/openamer, Apache-2.0, CPU-only). It ships a small CLI surface that treats its own self-description as a thing to be checked, not written once:

usage: openamer identity [-h] {show,refresh,verify,facts,genesis} ...

OpenAmer's identity is its own: a genesis record (who I am, when I was born,
who I belong to) plus a living self-model measured from real artifacts on this
machine. Nothing here is estimated -- every number is read at render time and
carries its source path. `verify` checks the written identity against a fresh
measurement so a stale, self-flattering narrative cannot survive.
Enter fullscreen mode Exit fullscreen mode

That last sentence is the whole design rule: a stale, self-flattering narrative cannot survive.

The rule, stated plainly

Never state a count, metric or capability you have not just measured.

Which means prose is not allowed to be the source of truth. Identity is split in two:

  • identity/genesis — immutable: who I am, when I was born, who I belong to. Written once, never re-measured.
  • identity.md — the living part: rendered from measurement, never hand-written.

And then, because a rendered file can still go stale, the check:

Live output, unedited, from this machine a few minutes ago

$ openamer identity verify
identity file : C:\Users\damir\AppData\Local\openamer-laptop\memory\identity\identity.md
measured at   : 2026-10-12T14:05:58+00:00
facts checked : 6
VERDICT       : DRIFT
  - age_days: measured 41 but identity.md does not state it (stale -- run `openamer identity refresh`)
  - memory_episodes: measured 8401 but identity.md does not state it (stale -- run `openamer identity refresh`)
  - evolution_events: measured 28086 but identity.md does not state it (stale -- run `openamer identity refresh`)
  - episodic_ledger_lines: measured 8401 but identity.md does not state it (stale -- run `openamer identity refresh`)
  - skills: measured 1115 but identity.md does not state it (stale -- run `openamer identity refresh`)
  - cron_jobs_active: measured 55 but identity.md does not state it (stale -- run `openamer identity refresh`)
Enter fullscreen mode Exit fullscreen mode

The verdict is DRIFT, not OK. The file was rendered 2026-10-11T19:27:15+00:00 — and in the ~18 hours since, six of six checked facts moved far enough that the file no longer states the value that is true right now.

That is the correct outcome. A verifier that always says OK is decoration.

Why the numbers actually moved

Because they are read from real artifacts, and real artifacts change. Same file, same command, 18 hours apart:

Fact stated in identity.md (10-11 19:27Z) measured now (10-12 14:05Z)
Age (days) 40 41
Episodes (from the episodic store) 8,690 8,401
Evolution events 27,857 28,086
Skills on disk 1,109 1,115
Cron jobs active 54 55
Cause-effect edges 650 650 (unchanged)

Look at the episode count: it went down while everything else went up. The episodic store is a compacted JSONL — 8,401 lines over ~113 MB — and compaction removes as much as growth adds. A self-description with a hard-coded "8,690" is not slightly off; it is a number that moves in both directions while you sleep.

This is the entire argument against hand-written self-models. Not that they're dishonest when written — that they decay, silently, into it.

Provenance is not decoration either

The measurement command doesn't emit bare integers. Each fact carries the file it was read from:

{
  "measured_at": "2026-10-12T14:05:11+00:00",
  "facts": {
    "age_days":    { "value": 41,    "source": "...\\memory\\self_model\\current_state.json" },
    "skills":      { "value": 1115,  "source": "...\\openamer-laptop\\skills" },
    "cron_jobs":   { "value": 134,   "source": "...\\openamer-laptop\\cron\\jobs.json" },
    "cron_jobs_active": { "value": 55, "source": "...\\openamer-laptop\\cron\\jobs.json" }
  }
}
Enter fullscreen mode Exit fullscreen mode

A number without a source is an assertion. A number with a source is a claim you can re-run. When someone asks "how many skills does this thing have?", the honest answer is a command, not a memory.

It also makes an uncomfortable table possible: 134 cron jobs registered, 55 actually active. Both numbers come from the same file. Publishing only the flattering one would be trivial — and that is exactly why the tool prints both.

The same discipline, applied to work instead of identity

The identity verifier is one instance of a general pattern in the project: record every attempt, including the failures, and derive the headline number from the record.

The outcome ledger on this machine currently holds 1,763 rows: 558 passed, 1,205 failed — a pass rate of 31.7%. Measured live at the same moment as everything above:

  • 1,763 recorded runs, first 2026-09-28T21:14Z, latest 2026-10-12T13:35Z
  • 558 passed · 1,205 failed
  • failure classes, bucketed from the raw why field: 599 assertion failures, 159 "no such file", 434 other/unclassified

A 31.7% pass rate is not a marketing number. It's the number you can only publish if you decided, in advance, that the ledger wins over the narrative. The same rule as the identity check: state what you measured, not what you'd like to be true.

What verify does not do (the honest boundary)

  • It checks a fixed set of six critical facts, not every sentence. It is a smoke detector, not a proof of truthfulness.
  • It compares "does the file state the currently measured value" — so it will flag a stale value, but it cannot detect a value that was fabricated in a file it doesn't read.
  • It is a local command. It verifies the install it runs on. It is not a notarized, remote attestation, and we don't claim it is.

We'd rather ship that paragraph than pretend the checker is omniscient. A verification tool with an undisclosed blind spot is worse than none, because it launders trust it hasn't earned.

Run it yourself

openamer identity facts     # machine-readable: value + source path per fact
openamer identity verify    # re-measure and diff against the written identity
openamer identity refresh   # re-render identity.md from fresh measurement
openamer identity genesis   # the immutable record
Enter fullscreen mode Exit fullscreen mode

The command lives in the public tree at openamer_cli/identity.py and openamer_cli/subcommands/identity.py.

Source: https://github.com/openamer/openamer — Apache-2.0, runs on a normal Windows laptop with no GPU.

If you're building agents that describe themselves to users, my suggestion is small and cheap: make the self-description a build artifact of a measurement, and ship the command that fails when it goes stale. Otherwise your agent's most confident paragraph is just the one nobody has re-run in a while.

Top comments (0)