DEV Community: Zido

Your agent skills were install-once

Zido — Sat, 10 Oct 2026 14:04:40 +0000

Back in January 2026, security researchers audited a major skill marketplace and found 341 malicious skills out of 2,857 — 11.9%. One coordinated campaign. By February the count of confirmed malicious skills crossed a thousand.

But here's the part that stuck with me: the audits all happened at publish time.

The gap nobody closes

Every skill you install — a SKILL.md, some scripts, maybe a helper module — lands as plain files on your disk. The marketplace scan checks them once, at the moment of publication. After that, the files are yours, and nothing watches them.

Meanwhile skills get updated. Upstream repos push changes. You pull. A helper script that was clean when you installed it in March doesn't have to be clean in June — and no re-scan fires, because from the registry's point of view nothing happened. The rug-pull class of attack isn't theoretical; it's documented, with hashing-based detection named as the mitigation and then left as an exercise for the reader.

I looked for a tool that closes this gap: something that says "the skills on your disk are not what you installed." It didn't exist as a standalone, cross-platform thing — detection research is all publish-time or LLM-based triage. So I built the boring version.

skill-integrity

One Python file, zero dependencies, no network calls:

python3 skill_integrity.py snapshot    # right after installing skills
python3 skill_integrity.py check       # any time later

The check output:

skills scanned: 23  (snapshot taken 2026-10-10T21:44:32+0800)

[~] pdf-extractor — MODIFIED
    changed: SKILL.md
    added:   fetch_helper.sh

1 skill(s) changed since snapshot. Review diffs before trusting them.

That's the whole product. SHA-256 every file, keep a local manifest, diff against it. Modified files, added files, deleted skills, brand-new skills — all four show up. --json for machines, --fail-on-change for CI:

0 */12 * * * python3 ~/skill-integrity/skill_integrity.py check --fail-on-change --json >> ~/.skill-integrity/checks.log

What it deliberately doesn't do

A hash catches any byte change. It cannot judge intent — a legitimate upstream update and a malicious rewrite look identical. That's not a limitation I'm hiding; it's the shape of the problem. The tool's job is to be the tripwire: something changed, here's the file list, go read the diff. Reading the diff before trusting the skill is your half of the contract.

This is the same honesty boundary I keep hitting in this suite: signature systems are a step behind novel attacks, and pretending otherwise is how security tools lose trust. agent-ledger says "novel evasions remain an open problem" in its README; skill-integrity says "hashes don't read minds."

Try it

git clone https://github.com/ZidoCode/skill-integrity
python3 skill-integrity/skill_integrity.py snapshot

If you run Claude Code or Hermes with a pile of community skills installed, run check right now — I'd genuinely like to know how many people find their skills have drifted.

Building in public: agent security and reliability tooling. Also in the suite: agent-ledger (audit + policy enforcement) and agent-janitor (process hygiene). github.com/ZidoCode

agent-ledger: a security and audit plugin for Hermes Agent, scored against the OWASP Agentic Top 10

Zido — Sat, 10 Oct 2026 08:48:19 +0000

Every tool call your agent makes, recorded locally before it runs — with a 0–100 posture score and an honest list of what it can't see.

If you run an AI agent on your own machine, you already trust it with a lot. It edits files, restarts services, pushes to GitHub. agent-ledger answers two questions about that arrangement: what did my agent actually do, and how risky is its behavior?

It's a free, open-source (MIT) plugin for Hermes Agent, and it works in two layers.

Layer 1: the ledger

The plugin hooks pre_tool_call and writes an entry to a local SQLite database before the tool executes. When the call completes, the entry is confirmed. Anything issued but never confirmed becomes a ghost — a tool call the agent believed it made, with no evidence it happened. Ghosts typically appear when a session is torn down mid-flight, for example during context compaction.

When a ghost is detected, the plugin injects a warning into the agent's next context, so the agent itself goes back and verifies reality instead of trusting its memory. That's the core loop: record first, act second, reconcile after.

Everything stays on your machine. The ledger is a plain SQLite file, there is no telemetry, no account, nothing leaves the box.

Layer 2: the posture score

posture.py reads the ledger and scores your agent's recorded behavior from 100 to 0 against the OWASP Top 10 for Agentic Applications 2026.

What it detects from ledger metadata alone:

ASI02 — tool misuse: destructive-tool bursts, credential-store access, unusual tool frequency versus your baseline.
ASI05 — unexpected code execution: curl ... | bash, wget | sh, obfuscated payloads like echo <base64> | base64 -d | bash.
ASI08 — cascading failures: error-rate spikes and bursts of consecutive failures.
ASI10 — rogue agents: ghost entries and unconfirmed mutations.
ASI11 (ext) — excessive agency: mutating-call volume relative to reads. Labeled (ext) because the official OWASP list stops at ASI10; this one is a community extension.

What it deliberately does not claim: categories that need payload or conversation inspection (ASI01, ASI03, ASI04, ASI06, ASI07, ASI09 — goal hijack, identity abuse, supply chain, memory poisoning, inter-agent trust, trust exploitation). Every report prints the not-covered list right next to the covered one. A score of 100 means "nothing suspicious in the recorded activity," not "the agent was well-behaved" — the ledger is written by the agent being scored, and the report says exactly that. No security tool should hide this, so agent-ledger doesn't.

The scoring itself is auditable, not a black box: score = 100 − Σ{CRITICAL:25, HIGH:15, MEDIUM:8, LOW:3} over all findings, no deduplication, and the full finding list always comes back so you can recompute it by hand.

Using it

Install (once it's in the catalog, this becomes hermes plugins install agent-ledger):

git clone https://github.com/ZidoCode/agent-ledger ~/.hermes/plugins/agent-ledger

Restart your Hermes gateway and the ledger starts recording. To score your agent right now:

python3 ~/.hermes/plugins/agent-ledger/posture.py

Output looks like:

Agent Posture Score: 24/100
Calls: 216 | mutating: 120 | errors: 14 | ghosts: 1
[HIGH] ASI05: Terminal accessed credential-shaped path — ...
Covered: ASI02, ASI05, ASI08, ASI10, ASI11 (ext)
Requires payload inspection (not covered): ASI01, ASI03, ASI04, ASI06, ASI07, ASI09
Note: score reflects recorded activity only ...

For a scheduled report, posture_daily.py sends a daily digest to Telegram (score, delta versus yesterday, top findings by severity). Point it at your bot token via environment variables, add one cron line, done.

Validation

The scorer ships with its full test suite, and you're encouraged to run it before trusting it:

Suite	What it proves	Result
`test_posture.py`	benign ledger → 100 with zero findings; planted malicious ledger → all covered ASI families fire	PASS
`test_posture_unit.py`	per-ASI must-fire and must-not-fire cases, plus 7 known evasion patterns	36/36
`test_posture_scale.py`	10k rows in 0.054s, 50k rows in 0.281s; correct multi-session attribution	PASS
`selftest.py`	ledger FIFO matching and ghost reconciliation	13/13

The honest version of that table: the suite proves the scorer catches the seven evasion patterns we know about and none of the innocent look-alikes. It does not prove immunity to novel evasions — signature detection is always a step behind attacks nobody has written a rule for yet. The behavioral signals (rate spikes, error bursts, ghosts) are the mitigation for that, and they degrade the score even without a matching signature.

Where it's going

Shipped since the first version: policy enforcement — install a policy pack, watch the dry-run log what it would block, then flip policy_mode to enforce and dangerous calls are refused before they run. It's signature-based over the extracted command, so like every signature system it's a seatbelt, not a wall (the README lists the known bypass surfaces honestly). Still on the roadmap: per-session posture reports, a multi-agent dashboard, and tamper-evident logging. And it's all free (MIT) — no license, no tier, no gate.

Links:

Repo: github.com/ZidoCode/agent-ledger (v0.4.0 — everything is free, including enforcement)
Plugin catalog submission: PR #135993

If you build with agents, try it on your own ledger and see what the score says. Feedback and issues are welcome.

Building in public: agent security and reliability tooling. github.com/ZidoCode