What happens when you delete 30 cron jobs and replace them with one heartbeat

Most "AI agent" stacks I've seen share the same skeleton: a pile of scheduled
tasks, one process per job, and a model that gets asked to be clever. It works
right up until you have thirty of them, and then you can't tell whether anything
is alive.

We run a small fleet of long-lived agents on one laptop. This is a write-up of
the one architectural change that mattered most: collapsing per-task scheduling
into a single subsystem heartbeat, and moving agent-to-agent traffic onto a
brokerless mesh where every message is verified before a handler runs. No
framework, no cloud. The whole thing is in-process.

The shape: one heartbeat, many subsystems

The old design was cron-per-task. Each job had its own schedule, its own
process, its own failure mode, and its own silence. When something broke you
found out because it stopped emailing you — which is indistinguishable from
"nothing to report."

The new design has one heartbeat. It dispatches a fixed set of subsystems,
each with its own cadence and back-off:

[ON] OK a2a             every 240m   Agent-to-Agent — brain export & peer comms
[ON] OK a2a_server      every 15m    A2A protocol server — realtime peer comms
[ON] OK cognition       every 30m    world model + prediction + self model, one pass
[ON] OK darwin          every 15m    evolutionary skill evolution
[ON] OK desktop         every 1440m  desktop evidence ledger — integrity check
[ON] OK infra           every 5m     browser, ssh, plugins, sync
[ON] OK learning        every 7m     internet + active + self-model learning
[ON] OK meta            every 30m    reflection, rewriting, research
[ON] OK outreach        every 180m   social & github outreach
[ON] OK security        every 240m   cve scan, bugbot, pen-test
[ON] OK senses          every 30m    senses & biological rhythm
[ON] OK swarm           every 30m    multi-agent coordination
[ON] OK system          every 5m     self-healing, resources, cache

Thirteen subsystems, one tick loop. The property you get back is visibility:
one place that can answer "what fired, what's due, what's been failing" instead
of a scavenger hunt across a scheduler. A task that starts throwing is isolated
and backed off rather than quietly starving its neighbours.

This is a real reduction, not a rebrand: the same machine runs 55 scheduled
jobs (53 active), and the heartbeat is what decides which of the agent's
internal loops actually run on any given tick. The jobs didn't disappear; the
silence did.

Five tools, no subprocess

The agent's own capabilities are exposed as exactly five native tools:

asi_status     # identity, episode count, world-model edges, evolution events
asi_think      # recursive reasoning loop
asi_learn      # internet learning cycle
asi_remember   # episodic recall over a causal (cause -> effect) store
asi_trigger    # fire a capability: self_improve, predict, validate, meta_learn, ...

The important word is native. Each tool imports its subsystem directly (e.g.
from tools.asi.a2a import run) instead of shelling out. Subprocess-per-capability
looks harmless until you have several loops running at once — then you're paying
for process spawn on every tick and losing the ability to share state. In-process
tools share the same world model, the same memory, and the same lock.

Measured on this box right now:

• 8,579 episodic memories (cause -> effect pairs)
• 659 world-model edges
• 28,167 evolution events

Those numbers move constantly, which is exactly why the status tool re-reads
them from live artifacts at call time instead of trusting a cached "about me."

The mesh: verify who spoke before acting

The part I'd defend hardest is the A2A (agent-to-agent) layer. The naive version
is "agents POST JSON to each other's HTTP ports." That falls apart the moment two
peers share a display name or one of them is compromised — "the scheduler told me
to" becomes unverifiable, and a poisoned agent can talk a healthy one into acting.

What we landed on is brokerless and boring:

• Identity is a key, not a label. Every message is signed by the sender and verified by the receiver before any handler runs. There's no central router to trust, and nothing to lose if one peer goes down.
• The relay is dumb on purpose. Peers drop signed envelopes into a shared relay; whoever is listening picks them up and validates before dispatching. No broker to become the single point of both coordination and failure.
• Capabilities, not credentials. An agent holds a scoped token (write:/tmp/x, call:send_email), not a long-lived secret. The check happens at the tool boundary, where the effect actually occurs — so an injected agent has nothing to present, and no prompt can mint a new token.

Why this over a framework

We didn't set out to avoid frameworks; we set out to run on one laptop without a
control plane. What that forced:

1. State in files, not context. A long-running agent reads its last durable state and last decision, acts, writes new state. The transcript is for humans.
2. A ledger for "did I already do this?" Append the intended effect before the side effect, keep the row only if it's durable, so a crash-resume no-ops instead of double-sending.
3. Verification as an artifact. "Done" is a query over recorded results — a manifest of (command, expected output, exit code) executed by something other than the agent that claims success.

None of that is about a smarter model. It's about giving a process a memory, a
brake, and a way to be checked.

---

If you run long-lived agents in the open, I'd like to hear how you handle the
"stuck for a week vs failed 40 times since Tuesday" distinction — that gap is
where most unattended stacks quietly die. The code is open source (Python, no
cloud required): github.com/openamer/openamer.

Comments

[Arjun Sharma]

The heartbeat-as-single-visibility-point generalizes beyond agents: cron-per-task fails the same way in any fleet, where silence and health are indistinguishable. On your closing question about stuck-for-a-week versus failed-40-times: I have found that tracking consecutive-failure count per subsystem plus time-since-last-success as two separate signals beats either alone. The count catches the flapping task, the timestamp catches the one that died quietly on the first try.

[Reid Marlow]

The way I split those two is separating the subsystem ledger from an external watchdog clock.

A subsystem failing 40 times is visible because every tick records an exit status and error string to a local table. Five consecutive non-zero exits trigger an exponential backoff that caps at four hours, and masks that specific loop so the rest of the tick keeps cycling.

The silent freeze is the harder bug: the process stays alive, holds a lock or socket, and never throws an exception. To catch that, the heartbeat runner writes a timestamp to a local file before each subsystem pass. A separate watchdog checks the delta on that timestamp every five minutes. If the delta drifts past fifteen minutes, it sends SIGKILL, cleans the pidfile, and logs a stall. The agent never evaluates whether it is hung; an external timer makes that call.