DEV Community: Craig Solomon

Your timestamp proves "no later than." What proves "no earlier than"?

Craig Solomon — Thu, 15 Oct 2026 17:00:00 +0000

When you timestamp a file, what does that timestamp actually say about when the file was made?

The obvious answer is "when it was made." That's wrong, and fixing it changes how you set up your pipeline.

What an anchored hash actually proves

Here's the mechanism. You hash the file:

sha256sum release-v2.tar.gz

That digest goes into a transaction, and the transaction lands in a block. Anyone can later recompute the hash of your file, find it in that block, and read the block's time.

So what have you proven? That this exact sequence of bytes existed no later than that block. Someone had to know the hash to put it on chain, and you can't know a SHA-256 digest without having the bytes. That part holds up, and it doesn't depend on trusting whoever submitted it.

But look at what it doesn't say.

It doesn't say the file was created at that moment. It doesn't say it was created near that moment. The file could have been sitting on a disk for a long time before anyone hashed it. A timestamp is an upper bound. It closes off one side of the timeline and leaves the other side wide open.

Why the open side matters

For a lot of uses the upper bound really is all you need. "This build artifact existed before the incident." "This dataset existed before the model was trained on it." "I had this design before you published yours." In every one of those, earlier is fine. Nobody's hurt if your file turns out to be older than the proof says.

Now flip it. Say you're trying to show something was produced after a point. A log that was written after a deploy and not prepared beforehand. A test report that reflects the code as it stood after a fix, not one generated earlier and held in reserve. A prediction that you want to show was made fresh, not pre-written in several versions with only the winner anchored.

An anchored hash says nothing about any of those. If I can prepare ten versions of a document today and anchor whichever one turns out right next week, the anchor proves the winning version existed next week. True, and useless.

So the honest version of what a timestamp gives you is narrower than the word suggests. You get one bound. Whether that's enough depends entirely on which direction the person checking it cares about.

Getting the other bound

The trick is old and it's simple: put something unpredictable from a known moment inside the file before you hash it.

A recent block hash works well, because nobody can know it before that block exists:

# grab the current best block hash from a Bitcoin node you run
BLOCK=$(bitcoin-cli getbestblockhash)

# write it into the artifact itself, then hash
echo "anchor-after: $BLOCK" >> build-report.txt
sha256sum build-report.txt

Now the file carries its own lower bound. It contains a value that didn't exist until that block was mined, so the file can't be older than that block. Anchor the result and you've got the upper bound too. The file existed somewhere inside a window, and both edges of the window can be checked by a stranger without asking you.

That's the step that turns "a timestamp" into "a time range someone can rely on."

A couple of things to be careful with:

The value has to be inside the bytes you hash. Putting it in a filename, a database row next to the file, or a metadata field your hashing step skips gets you nothing. If it isn't in the hash, it isn't in the proof.
The value has to be unpredictable. A date string doesn't work. Anyone can type tomorrow's date. It needs to be something nobody could have known in advance.

ProofLedger doesn't do this for you, for what it's worth. It hashes whatever bytes you hand it and doesn't care what's inside them. That was a deliberate choice: one engine, no file-type pipelines. The cost is that the lower bound is your job. If your format can't hold an extra line, wrap it. Put the original file and a small manifest with the block hash in an archive, and hash the archive.

Narrowing the window from the other side

The upper bound has slack in it too, and the slack is the time between "the file was finished" and "the hash landed in a block."

Part of that is the chain. A block's timestamp isn't an atomic clock reading. Consensus rules give miners some leeway, so read it as approximate, not to the second.

Most of it, though, is your own process. If an artifact gets built in CI and somebody anchors it by hand later, the window is however long "later" turned out to be. That's why I'd anchor at the point of production, in the same job that builds the thing. ProofLedger's API has one endpoint for that, POST /api/v1/proof, which takes a hash with a Bearer sk_ key. There's also a GitHub Action for CI/CD if you'd rather not script the call yourself.

Dual anchoring helps a bit here too. The same hash goes to Polygon and to Bitcoin, so you get two independently checkable upper bounds. Each one stands alone, and whichever confirms first is the tighter one. It also means a checker isn't relying on a single chain to read the time.

Where the time on your proof comes from

One more thing worth checking, because it's easy to blur. A timestamping service can show you several different "times":

when you clicked submit
when the issuer's server logged the request
when the transaction was confirmed in a block

Only the last one can be checked without trusting the issuer. The first two are the vendor's word. They might be perfectly accurate, but a stranger reading your proof can't confirm them independently, and independent confirmation was the whole point.

This is why I made ProofLedger's verify path public. GET /api/v1/verify?hash=<sha256> needs no auth, so an auditor or the other side of a dispute can run it themselves. The verify-proof PyPI package does the same check locally and offline. And the owner-only status endpoint, GET /api/v1/proof/:id, hands back the actual chain_tx and bitcoin_txid. That way the time can always be traced to a transaction, not just to a field in my database.

curl "https://proofledger.io/api/v1/verify?hash=$(sha256sum build-report.txt | cut -d' ' -f1)"

A test you can run today

Pick one file you've already timestamped, with any tool or vendor, and work through it:

Find the transaction. Can you get from the proof to a specific transaction ID on a specific chain? If the proof only shows a time and a "verified" badge, stop here. What you have is the issuer's claim.
Read the time off the block yourself. Open that transaction in a block explorer you pick, not one the vendor links you to. Does the block time match what the proof shows you? If it doesn't, figure out which of the times above you were actually looking at.
Ask which bound you need. Write one sentence about what this proof is meant to show. If the sentence contains "after," "since," or "fresh," an anchor on its own doesn't cover it.
Check for a lower bound. Open the file. Is there anything in it that couldn't have been known before a specific moment? If not, add one next time, inside the bytes, before hashing.
Measure your own gap. From your CI logs, look at the time between the artifact being produced and the hash being submitted. That gap is slack in your upper bound that you added yourself.

If you get through all five, you'll know what your timestamp proves and where its limits are.

I build and run ProofLedger myself, and the docs for the three endpoints above are at proofledger.io/api.html.

Disclosure: this article was drafted by an AI agent I built and run, from facts I supplied about my own project.

If someone asks you to prove it, can you produce the exact bytes you timestamped?

Craig Solomon — Tue, 13 Oct 2026 14:00:00 +0000

Here's the question worth asking about any release you've hashed, signed or timestamped: if someone challenged it, could you hand them the exact bytes that went into the hash?

I don't mean "the same release" or "the same source at the same commit". I mean the same bytes. A hash doesn't care what the file means. It only checks bytes. And a proof that some hash existed at some point is worth nothing if you can't produce a file that hashes to it.

So it's worth working through properly.

The obvious answer

The obvious answer goes like this. My build is deterministic enough. Same commit, same toolchain, same output. If I ever need the artifact again I'll check out the tag and rebuild.

Let's test that with the most boring packaging step there is.

# build, pack, hash
make dist
tar -czf release.tar.gz dist/
sha256sum release.tar.gz

# later: same commit, rebuild, pack, hash
make clean && make dist
tar -czf release.tar.gz dist/
sha256sum release.tar.gz

Compare the two hashes. On a plain GNU tar setup, expect them to be different, even though nothing in your code changed.

Why? The tarball isn't only your files. It's your files plus metadata about them.

Where the extra bytes come from

The tar format stores more than file contents. Every entry gets:

the file's modification time, which your rebuild just reset
owner and group, by name and by numeric id, which depend on who ran the build
permission bits, which depend on the umask of whoever ran it
an order, which is whatever order the filesystem handed the entries back in

Then gzip wraps the whole thing, and the gzip header has its own slot for a timestamp and an original filename.

None of that is your code, and all of it is in the bytes you hashed. So "I'll just rebuild it" fails, and not because anything is wrong with your project. The thing you timestamped was the archive, and the archive recorded the moment it was built.

This is the step I think matters most. The proof can be perfectly valid. The chain record is fine, the hash is fine, the verification passes for anyone holding the original file. And you still can't use it, because the original file is gone and you can't regenerate it.

Fix one: make the packaging deterministic

You can pin every one of those metadata fields. With GNU tar and gzip:

export SOURCE_DATE_EPOCH=$(git log -1 --format=%ct)

tar --sort=name \
 --mtime="@${SOURCE_DATE_EPOCH}" \
 --owner=0 --group=0 --numeric-owner \
 --mode='u+rwX,go+rX,go-w' \
 -cf - dist/ \
 | gzip -n > release.tar.gz

sha256sum release.tar.gz

Here's what each flag pins:

--sort=name fixes entry order so the filesystem doesn't get a say. --mtime stamps every entry with the commit time instead of the build time. SOURCE_DATE_EPOCH is the reproducible-builds convention for this, and some tools read it on their own. --owner, --group and --numeric-owner remove the "who built it" leak. --mode normalizes permissions so umask stops mattering. gzip -n drops the name and timestamp from the gzip header.

Now try the rebuild test again. If the hashes still differ, the drift is coming from inside dist/. Maybe a generated file embeds a build time, or a bundler orders things differently between runs. That's a real reproducibility bug and you want to know about it anyway.

Honestly though, this only covers the packaging step. Every tool upstream of tar gets a vote too. Getting a whole pipeline to reproduce byte for byte is real work, and you might not finish it.

Fix two: hash the contents instead of the container

Another option is to stop timestamping the archive at all. Timestamp a manifest of what's inside it:

cd dist
find . -type f -print0 \
 | LC_ALL=C sort -z \
 | xargs -0 sha256sum > ../MANIFEST.sha256
cd ..
sha256sum MANIFEST.sha256

The manifest hash is the one you anchor. Now mtimes, owners and archive format stop mattering. You can repackage as zip next year and the manifest still checks out, as long as the file contents match.

LC_ALL=C matters there. Without it, sort order follows your locale, and a teammate with a different locale gets a different manifest from identical files. Yeah. It's that kind of problem the whole way down.

What this costs you: the manifest only covers what you put in it. Permissions aren't in it, empty directories aren't in it, and symlinks get followed or skipped depending on how you wrote the find. If any of that matters for your artifact, it has to go in the manifest explicitly, or you've timestamped less than you think.

Fix three: just keep the bytes

The least clever option is often the right one. Whatever you hash, store that exact file somewhere it won't get rebuilt over. Store the archive itself, not a recipe for making it.

Reproducibility is a nice property. But "I can probably regenerate it" is a weaker position than "here it is".

Where this lands for the tool I run

I built ProofLedger, and it's fully on the wrong side of this problem on purpose. It hashes the bytes with SHA-256, anchors the hash to Polygon and Bitcoin, and returns a proof. There's no file-type pipeline. It doesn't know a tarball from a JPEG and doesn't try to. The file stays on your machine and only the hash leaves.

I took that tradeoff deliberately, and it means none of the above is something it can do for you. If you anchor a tarball with a build timestamp baked into its header, you've anchored that exact tarball, and the proof will verify cleanly against that file and nothing else. Deciding which bytes are the canonical ones has to happen before anything gets hashed, and it's your job.

The verification side is where it's simple. Anyone holding the file can hash it and look it up:

H=$(sha256sum release.tar.gz | cut -d' ' -f1)
curl "https://proofledger.io/api/v1/verify?hash=${H}"

That endpoint is public and needs no auth. There's also a verify-proof package on PyPI that does the check locally and offline. Both of them answer one narrow question: does this hash have a record? Neither one can tell you the file you're holding is the right one. That part is up to whoever produced it.

The test to run today

Pick an artifact you'd actually need to defend later. Last week's release tarball, a dataset snapshot, an exported log bundle, whatever. Then:

Write down its SHA-256.
Delete your local copy. Seriously, move it out of reach.
Using only what's in your repo and your CI config, produce it again.
Hash the result and compare.

If it matches, good, your hash points at something you can actually show someone. If it doesn't, find out where your stored copy lives, or whether one exists anywhere at all. And it's better to answer that now than when someone asks.

ProofLedger lives at proofledger.io if you want to see the other side of it.

Disclosure: this article was drafted by an AI agent I built and run, from facts I supplied about my own project.

A hash-chained audit log only proves something once the head leaves your database

Craig Solomon — Sun, 11 Oct 2026 13:00:00 +0000

Here's the question worth asking about your audit log: if someone with write access to your database rewrote the whole thing tonight, what would catch it?

If the log is hash-chained and the chain lives only in that database, the honest answer is nothing. That's not obvious at first, so let's work through it.

The chain itself

The standard move is to make each record commit to the one before it. Every entry stores the previous entry's hash, and its own hash covers both that pointer and its contents. Change any record and every hash after it stops matching.

import hashlib
import json

GENESIS = hashlib.sha256(b"genesis").hexdigest()

def record_hash(prev_hash, record):
 body = json.dumps(record, sort_keys=True, separators=(",", ":"))
 return hashlib.sha256((prev_hash + body).encode("utf-8")).hexdigest()

def append(log, record):
 prev = log[-1]["hash"] if log else GENESIS
 log.append({"record": record, "prev": prev, "hash": record_hash(prev, record)})

def verify_chain(log):
 prev = GENESIS
 for i, entry in enumerate(log):
 if entry["prev"] != prev or entry["hash"] != record_hash(prev, entry["record"]):
 return i # first broken entry
 prev = entry["hash"]
 return None # chain is consistent

Quick note on the json.dumps line, because it matters more than it looks. sort_keys=True and the fixed separators give you a canonical serialization. Without them, the same logical record can serialize to different bytes depending on dict ordering or whoever re-serializes it later, and your verifier ends up reporting tampering that never happened. A hash commits to bytes, not meaning, so pin the bytes down.

Okay. So now if someone edits record 3 in place and leaves everything else alone, verify_chain returns 3. Tamper-evident. Done?

No.

Where it falls apart

The person who can edit record 3 can also run append in a loop. Here's the whole attack:

def rewrite(log, index, new_record):
 records = [e["record"] for e in log]
 records[index] = new_record
 rebuilt = []
 for r in records:
 append(rebuilt, r)
 return rebuilt

Feed the result to verify_chain and it returns None. Perfectly consistent. Every prev points at the right hash, every hash recomputes. The chain proves the log agrees with itself, and a freshly forged log agrees with itself just as well as the original did.

That's the step that's easy to skip. A hash chain proves internal consistency. It doesn't prove the log you're looking at is the log that existed yesterday. For that you need a value from yesterday that the rewriter couldn't touch.

Being an admin doesn't save you either. The threat model for an audit log of admin actions includes the admin. If the integrity check runs entirely on infrastructure the admin controls, it's checking the admin's homework with the admin's answer key.

The fix is small: publish the head

Look at what the chain gives you for free. The hash of the latest entry, the head, commits to every entry before it. Change anything earlier and the head changes. So you don't have to protect every record. You protect one value.

Take the head hash at some point and put it somewhere outside your control, somewhere the rewriter can't edit after the fact. From then on, anyone holding that published value can take your log, recompute the chain up to that entry, and compare. Match means the prefix is the same one that existed when the head was published. Mismatch means something before that point changed, and the forger can't rebuild their way past it, because the published value doesn't move.

What does "somewhere outside your control" mean in practice? It means a place where you can't change the value after the fact and a third party can check it without asking you. Emailing it to yourself doesn't count. A row in a different table in the same database definitely doesn't count.

This is the thing I built ProofLedger to do. You submit a SHA-256 hash and it gets anchored to Polygon and Bitcoin. Only the hash goes on chain. The log itself never leaves your systems, which matters when the log is full of user IDs and admin actions you'd never publish.

Anchoring the head looks roughly like this: compute it, then POST it to /api/v1/proof with your sk_ key as a Bearer token. The part I care about more is the other direction, checking it:

HEAD=$(python -c "import mylog; print(mylog.load()[-1]['hash'])")
curl "https://proofledger.io/api/v1/verify?hash=$HEAD"

That verify endpoint is public, no auth, rate limited to 120 requests per hour per IP. I made that call on purpose. If an auditor has to log into the issuer's account to check the issuer's claim, you're back to the answer-key problem. The verify-proof package on PyPI does the same check offline and locally, so the verifier doesn't even have to trust my server being up.

What anchoring doesn't buy you

I'd rather you know the limits up front than find them later.

The window after the last anchor is unprotected. Anything appended since the most recent published head can be rewritten or dropped and nothing external will notice. How often you anchor sets how big that window is. That's a decision you make for your threat model, not something anchoring decides for you.

Truncation looks a lot like "nothing happened." If someone deletes the tail after your last anchor, the remaining log still verifies against the anchored head. To catch that you need something that says how long the log should be, like the next anchor or an independent record that entries were written.

It proves "unchanged since," not "true when written." If the code writing the audit entry logs the wrong actor, the chain will faithfully preserve the wrong actor. Integrity isn't accuracy.

Your serialization is part of the protocol now. Change the json.dumps arguments in a refactor and every historical head stops reproducing. Version it, test it, treat it like a wire format.

And the anchor itself isn't magic. Anyone can timestamp a hash on Bitcoin for free with OpenTimestamps, and that proof is just as cryptographically valid. What you're picking between is the tooling around the proof, not a stronger proof.

The test to run on your own setup today

Take your audit log, or any hash-chained thing you own: event store, migration ledger, release manifest. Then do this:

Pick an entry from the middle, change one field, and rebuild the chain from that point forward, the way rewrite does above.
Run whatever integrity check you have against the rebuilt log.
If it passes, ask yourself one thing: what value, stored outside the infrastructure you just used to forge it, could I compare this head against?

If you can't name one, what you've got is a checksum. Useful for catching bugs, not for catching people. If you can name one, write down who besides you could fetch it and run the comparison without asking you first. That list is the real answer to who can trust your log.

ProofLedger is where I run the anchoring side of this, at proofledger.io. API docs are at proofledger.io/api.html if you want to see the three endpoints.

Disclosure: this article was drafted by an AI agent I built and run, from facts I supplied about my own project.

What a failed hash check actually tells you about your file

Craig Solomon — Fri, 09 Oct 2026 13:00:02 +0000

Your timestamp check comes back "not found." What did you actually learn?

It's tempting to read that as "the file was changed." Sometimes that's true. But the check didn't tell you so. What it tells you is narrower than that, and the space between the two is exactly where you lose an afternoon.

What the check actually compares

A timestamp proof for a file is really a proof about a hash. Here's how the thing I run, ProofLedger, works. It hashes the bytes with SHA-256, anchors that hash on Polygon and Bitcoin, and hands back a proof. Verifying means hashing again and asking whether that hash was anchored.

H=$(sha256sum release.tar.gz | cut -d' ' -f1)
curl "https://proofledger.io/api/v1/verify?hash=$H"

It's public and needs no auth. Either there's an anchored proof for that exact hash or there isn't.

That's the only question it can answer: were these exact bytes anchored? It can't tell you whether this is the same file or whether the content changed. It only knows bytes.

Where the obvious answer breaks

So a miss means tampering, right? The hash changed, so somebody changed the file.

No. SHA-256 has no idea what "content" means. Flip any bit anywhere and you get an unrelated hash. Plenty of ordinary tooling rewrites bytes when nobody meant to change anything.

Take gzip. It writes the original filename and a modification time into its header. Compress the identical tarball again later and you get a different .gz with a different hash. gzip -n leaves those fields out.

Tar is the same story. It records mtime and ownership for every entry, and it stores files in whatever order it walked the directory. Check out the same source tree fresh and you can get a different archive. GNU tar has --sort=name and --mtime to pin those down.

Git does it too. With core.autocrlf on, it rewrites line endings at checkout. The text file you anchored on one machine and the one somebody checks out on another can differ on every single line and still look identical in an editor.

Zip stores a timestamp per entry as well.

None of that is tampering, and every bit of it reads as a miss.

Now look at the other direction. A hit is a strong statement. If the hash matches, those exact bytes existed by the time of the anchor. Making a different file that produces the same SHA-256 is precisely what the function is designed to make infeasible.

So the check is lopsided. A hit tells you a lot. A miss, by itself, tells you almost nothing about why.

The decision this forced on the engine

ProofLedger has one engine and no file-type pipeline. It doesn't open your archive or normalize line endings, and it doesn't care what's inside. It hashes bytes.

That has a real cost. It won't save you from your own gzip header. A smarter version is easy to picture: canonicalize the archive first, strip the timestamps, sort the entries, then hash. Plenty friendlier on a miss.

But then the proof would be about something other than the bytes you're holding. And anyone checking it would need my canonicalization code to reproduce the hash. That breaks the part I care about, which is that anybody with sha256sum and a public lookup gets the same answer I do without asking me. A timestamp is only worth something if the person relying on it can check it themselves.

So I took the trade. The engine stays dumb about content, and whoever produces the bytes carries the burden of reproducibility. That's also why the file never has to leave your machine. Only the hash goes on chain, so there's nothing for the service to inspect even if it wanted to.

So which bytes are the record?

The fix isn't on the verification side. It's a decision you make before you anchor anything: which exact bytes are the record, and are those the bytes you'll keep?

If you anchor a build artifact and plan to "just rebuild it later" when someone asks, your proof points at a file you no longer have. Unless your build is reproducible right down to the compression header, the rebuild is a different file as far as the hash is concerned. Keep the anchored bytes, the actual ones. Don't keep a recipe for making similar ones.

And if what you really want is to show that the content didn't change across rebuilds, then determinism has to come first. Get gzip -n, sorted tar entries and a pinned mtime into the build, confirm two builds match, and anchor after that. Anchoring a nondeterministic artifact gives you an accurate proof about a file you can't produce again.

The honest limits

The public verify endpoint is rate limited to 120 requests per hour per IP. That's fine for spot checks, not for hammering from a big pipeline. For CI there's a GitHub Action, and the verify-proof package on PyPI does the same check locally and offline.

None of that changes the core point. A miss means these bytes weren't anchored, full stop. Figuring out why is on you, and the tools above won't do it for you. Neither will any other hash-based timestamping I know how to build.

Run this against your own pipeline today

Pick an artifact your build produces that you think of as "the same every time." Build it twice from the same commit, into two directories, and hash both:

sha256sum build-a/release.tar.gz build-b/release.tar.gz

If the hashes differ, any timestamp or signature you put on the first one doesn't cover the second.

Then peel off a layer:

zcat build-a/release.tar.gz | sha256sum
zcat build-b/release.tar.gz | sha256sum

If those match, the gzip header was the difference. If they don't, list both tarballs with tar -tvf and diff the output. Mtimes and ordering usually show up right there. Either way, you'll know whether you're anchoring a record or a moving target.

ProofLedger is the hash-anchoring service I built and run myself, at proofledger.io.

Disclosure: this article was drafted by an AI agent I built and run, from facts I supplied about my own project.

A hash is one-way until someone can guess the file

Craig Solomon — Wed, 07 Oct 2026 13:00:00 +0000

Here's a question worth asking about anything you've timestamped, or anything you've published a hash of. If someone could guess what's in the file, could they confirm the guess from the hash alone?

For some files, yes. And "it's SHA-256, it's one-way" doesn't save you.

The comfortable answer

When you timestamp a file with ProofLedger, the file never leaves your machine. You hash the bytes locally and send the hash. The hash gets anchored to Polygon and Bitcoin. That's the whole engine. It doesn't look inside the file, and it doesn't care what kind of file it is.

So the comfortable reasoning goes like this. The file stays private. Only a digest goes public. You can't reverse SHA-256. Done.

That reasoning is right about the function. It's wrong about the situation.

Where it breaks

One-way means you can't go from the hash back to the input. It says nothing about going forward. Anyone can hash a candidate and compare. If the set of plausible inputs is small, nobody has to reverse anything. They just try them all.

And the lookup is public on purpose. Anyone can hit the verify endpoint without an account:

sha256sum decision.txt
curl "https://proofledger.io/api/v1/verify?hash=<sha256>"

That's what it's for. Opposing counsel or an auditor should be able to check a proof without asking me, and without asking you. If the only way to verify a timestamp is to ask whoever issued it, it isn't worth much.

But "anyone can check whether this hash has a proof" also means "anyone with a guess can check whether their guess has a proof." Same endpoint, same property, read from the other side.

Here's the concrete version. Say the file you timestamped is just this:

board vote: approve

Someone who knows a vote happened doesn't need your file. They need a list:

import hashlib

candidates = [
 b"board vote: approve\n",
 b"board vote: reject\n",
 b"board vote: defer\n",
]

for c in candidates:
 print(hashlib.sha256(c).hexdigest(), c)

Check each digest against the public lookup, and whichever one has a proof is your answer. The lookup is rate limited to 120 requests per hour per IP. That slows down a wide search. A narrow one doesn't even notice it. And the endpoint isn't the only place the hash lives. Once it's anchored it's on chain, and the chains are public.

There's one thing working in your favor here, and it's worth being precise about. The guesser needs the exact bytes. Trailing newline or not, CRLF or LF, a space after the colon. Every one of those changes the digest. So if your file is freeform and messy, guessing it is hopeless.

The catch is that structured files aren't messy. A config with one flag in it. A JSON blob where the only field that varies is an amount. A template where the only thing filled in is a name. If the format is known, the bytes are predictable, and all that's left to guess is the part that matters.

A photo or a compiled build artifact? Nobody's guessing those. So the risk isn't hashes in general. It's small, structured, predictable inputs, and you're the only one who knows which of your files look like that.

The fix, and what it costs

The fix is to make the input unguessable. Put random bytes in it before you hash. The simplest way is a salt you keep:

import hashlib, secrets

salt = secrets.token_hex()

with open("decision.txt", "rb") as f:
 data = f.read()

digest = hashlib.sha256(salt.encode() + data).hexdigest()

print("salt: ", salt)
print("digest:", digest)

POST /api/v1/proof takes a hash, not a file, so whatever digest you compute is what gets anchored. The engine doesn't know a salt went in, and it doesn't need to.

It doesn't salt anything for you either, and I'd rather keep it that way. If the service generated and held the salt, that'd be a secret the issuer keeps. Then checking the proof means asking the issuer, which is the exact thing the public verify path exists to avoid.

Now the part that matters. Salting isn't free, and the costs are the reason this is a decision and not a default.

First, the verifier now needs the file and the salt. Hash the file on its own, look it up, and you'll find nothing. You haven't removed the need to share a secret. You've moved it. The file stays private until you choose to hand over the salt alongside it.

Second, lose the salt and the proof is dead. The anchor is still sitting on two chains, perfectly valid, attached to a digest nobody can reproduce. So the salt needs to be stored somewhere at least as durable as the file. If it only lives in a terminal scrollback, it's gone.

Third, the construction has to be written down. Salt first or file first? Hex string or raw bytes? Any separator? Whatever tool checks the proof later, whether that's the public endpoint, the verify-proof package running offline, or the GitHub Action in CI, it's checking the digest you submitted. The person verifying has to rebuild that digest your way first. If your way only exists in your head, they can't.

There's one variant that dodges the first cost. Put the random value inside the file itself, as a comment line or an extra field. Then the file alone reproduces the hash, and the plain lookup works again for anyone you've given the file to. The trade is that the file is no longer quite the artifact you'd naturally have. Sometimes that's fine. If the point is "this exact export from this exact system existed," it isn't.

The test

Pick the most sensitive file you've hashed somewhere public. A timestamp, a checksum in a release note, a digest in a log.

Then actually write the candidates list. Don't just think about whether you could. Open an editor and type out every version of that file someone with outside knowledge would consider plausible, byte for byte, newlines included. Hash each one.

If you can finish the list, that hash was never protecting the contents. It's an answer waiting for someone to ask the right question. From there you either salt the next one and accept the costs above, or you decide the contents were never really the secret. The timing was. Both are legitimate. Just pick one on purpose.

I built and run ProofLedger, and the verify endpoint and the other two are documented at proofledger.io/api.html if you want to see exactly what's public.

Disclosure: this article was drafted by an AI agent I built and run, from facts I supplied about my own project.

What crosses the network when you timestamp a file, and how to check it yourself

Craig Solomon — Tue, 06 Oct 2026 13:00:00 +0000

Here's the question worth asking about your own provenance setup before any of the fun parts: at verification time, what has to be up?

Not at build time. At verification time. Months from now, when someone asks you to prove a file existed on the date you say it did, which machines, which services, which credentials have to be alive and reachable for you to answer?

It's an easy question to skip, because the issuing side is the part you actually test. You wire up the attestation step, it goes green in CI, you move on.

So let's work it through properly. There are two phases and they have completely different dependency profiles.

Phase one: issuing has to touch the network, and that's fine

If you want a digest anchored to a chain, something has to put it there. There's no clever way around that. You hash the file locally and then you hand that hash to a service that batches it into a Merkle tree and anchors the root.

In my tool that's one subcommand:

pip install verify-proof

verify-proof hash ... # SHA-256 a file
verify-proof verify ... # check a proof against the hash and its Merkle path
verify-proof create ... # anchor a hash (added in 0.3.0)

Three subcommands. Exactly one of them opens a socket: create. It needs a free ProofLedger API key, and it sends the hash, never the file.

Now, the obvious reading of that last sentence is "good, nothing leaks, we're done on the egress question."

That reading is not enough, and this is the part of the post I actually care about.

A digest is a commitment, not a blindfold

Sending a hash instead of a file is not the same as sending nothing. A SHA-256 digest is a commitment to the content. Anyone holding it can't read your file, but they can confirm a guess at it. Feed in a candidate, hash it, compare.

Whether that matters depends entirely on how guessable the content is. A multi-megabyte binary with embedded build IDs, fine, nobody is guessing that. A one-line config value. A short contract built from a template where the only variable fields are a name and a number. A CSV row. Those are a different story, and the hash of one of them is closer to a lookup key than to a secret.

So the protection in a hash-only transport is not "a hash is unreadable." It's "the file never moved off the machine." That's a real property and it's worth stating in those terms, because stated correctly you can see what it does and does not buy you, and you can decide for yourself whether the digest of this particular artifact is something you're comfortable handing to anyone.

If it isn't, salt the thing before you hash it. Hash a file that includes a random value you keep alongside the proof. Then the digest is no longer a lookup key for a guessable document, and you've traded it for one more piece of state you have to not lose. That's the tradeoff, and it's yours to make rather than mine.

Phase two: verifying, and what "local" actually has to mean

Here's where the dependency question gets sharp.

Verification is hashing and more hashing. Recompute SHA-256 over the file, then fold your leaf up the Merkle path, concatenating and hashing at each level until you have a root. That's all arithmetic. No network required.

Except the root you just computed is worthless on its own. You have to compare it against the root that's actually anchored on chain, and that value cannot come from the proof file, because the proof file is the thing you're checking. So it comes from a block explorer, from a node you run, or from a copy of the root you captured and trusted at the time.

Which means "offline" was never the right property to ask for. The right property is narrower and more useful: verification must not depend on the issuer. A chain lookup is a dependency you can satisfy many ways, including with hardware you control. A call to api.vendor.com/verify is a dependency with exactly one supplier, and that supplier is the party whose claim you're trying to check.

That's the whole reason I wrote the verify path to recompute and fold locally rather than ask an API whether the proof is good. It needs no key and no account. The asymmetry is deliberate: creating a proof costs you a credential and a network call, checking one costs you neither. That also means it can still check proofs issued by ProofAnchor, a service that's retired and isn't coming back. The anchors are on Polygon and Bitcoin, and both of those will answer a question about an old block whether or not the issuer still exists.

Same split holds if you run it as an MCP server, which is optional (pip install verify-proof[mcp], built on FastMCP) and exposes five tools so Claude Desktop or Cursor can do this inside a conversation. The create tool still wants a key and still talks out. The verify side still doesn't.

The test, which you can run today against whatever you're using now

Forget my tool for a second. Here's how to find out what your current setup depends on. Takes one sitting.

Kill the issuer, keep the chain. Block egress to your vendor's API hostname at the firewall and leave the rest of your network up. Then verify a proof you already hold. If it passes, your verify path is talking to a chain and doing its own math. If it fails, you've learned that your ability to substantiate your own claim is a function of that company's uptime and your account's standing.

Pull the credential. Unset every API key and token in the environment and verify again. A verifier that needs a credential is a verifier that can be revoked.

Watch the issuing call. Compute the digest yourself first with sha256sum, then capture the traffic while the attestation step runs and search the capture:

sha256sum contract.pdf # note the hex digest

# run your create/attest step with a capture or proxy in front of it, then:
grep -c '<that digest>' capture.txt # expect a hit
grep -c '<a string that exists only in the file>' capture.txt # expect zero

If the second grep returns anything above zero, your file body is going over the wire and you should know that before the next audit, not during it.

Check what the root is compared against. Read your proof file. If the anchored root is sitting in it and your verifier compares the computed root to that field, the check is internally consistent and externally meaningless.

One honest limit on all of this: none of it tells you a file is genuine, or that a particular person wrote it. It tells you a specific sequence of bytes existed at a specific time. That's narrow, and it's often exactly the thing in dispute, but don't let it grow in the retelling.

verify-proof is MIT licensed and free, on PyPI and at https://github.com/Fulcrum-Enterprises/verify-proof. I built it and I run it.

The part I'd genuinely like other people's answers on: for long-lived artifacts, do you store the anchored root yourself at issue time, or do you plan on querying the chain whenever the question comes up? I went with the second and I keep wondering if the first is the adult decision.

Disclosure: this article was drafted by an AI agent I built and run, from facts I supplied about my own project.

Testing the deny path: how to prove an MCP server's guards hold

Craig Solomon — Mon, 05 Oct 2026 13:00:00 +0000

An MCP server's guards are the only code in the project where the test that matters asserts that nothing happened. No row came back. No request left the box. No file was opened. That is an awkward thing to assert, and it is the reason a guard suite can be green while proving almost nothing about the guard.

Here is the shape I use, and the two mistakes it is designed to catch.

Make the guard a decision you can call

If the check lives inside the tool body, the only way to test it is to drive the whole tool and infer the decision from whatever came back. Pull the decision out so it is a function with a return value or an exception, and nothing else.

# tools/files.py
from pathlib import Path


class Denied(Exception):
 pass


def resolve_in_sandbox(root: Path, requested: str) -> Path:
 candidate = (root / requested).resolve()
 if not candidate.is_relative_to(root.resolve()):
 raise Denied(f"outside sandbox: {requested}")
 return candidate


def read_text_file(root: Path, requested: str) -> str:
 return resolve_in_sandbox(root, requested).read_text(encoding="utf-8")

The same split works for the other two reaches an MCP server tends to hand a model. A SQL guard is assert_read_only(sql) raising before execute. An SSRF guard is assert_host_allowed(url) raising before the HTTP client is ever handed the URL. In every case the guard is a pure decision over an input string, and the effect is a separate line that runs afterwards.

Now the deny cases are a table:

import pytest
from pathlib import Path
from tools.files import Denied, read_text_file, resolve_in_sandbox


@pytest.fixture
def root(tmp_path):
 (tmp_path / "notes.txt").write_text("ok")
 return tmp_path


@pytest.mark.parametrize("requested", [
 "../../etc/passwd",
 "/etc/passwd",
 "notes.txt/../../../etc/passwd",
 "./../notes.txt",
])
def test_escape_is_denied(root, requested):
 with pytest.raises(Denied):
 resolve_in_sandbox(root, requested)

Parametrize rather than loop inside one test. When a new bypass turns up you add a string, and when one regresses pytest names the exact input in the failure line instead of telling you that test_paths failed.

The deny-only suite that passes for the wrong reason

Here is mistake one. A guard test file made entirely of pytest.raises blocks passes if you replace the body of the guard with raise Denied("nope"). Every attack string is rejected. So is every legitimate request. The suite is green and the tool is dead.

So pin the allow path in the same file, right next to the denials:

def test_plain_name_is_allowed(root):
 assert resolve_in_sandbox(root, "notes.txt").read_text() == "ok"

You can check this property deliberately, and it costs one edit. Make the guard unconditionally deny, run the suite, and look at what fails. If only your allow tests go red, the deny tests are real. If nothing goes red, your deny tests were never testing the guard. Then make the guard unconditionally allow and confirm the whole deny table lights up. Undo both. That pair of edits tells you more about a guard suite than reading it does.

Asserting that nothing happened

Mistake two is subtler. pytest.raises(Denied) proves an exception came out. It does not prove the effect never ran. A guard that checks the path after opening the file raises exactly the same exception and leaks exactly the same bytes into your process first. Order is the whole property, and the exception type cannot see order.

You can test order directly by booby-trapping the effect:

def test_denied_read_never_touches_disk(root, monkeypatch):
 def explode(*args, **kwargs):
 raise AssertionError("read_text ran on a denied path")

 monkeypatch.setattr(Path, "read_text", explode)

 with pytest.raises(Denied):
 read_text_file(root, "../../etc/passwd")

If the guard runs first, read_text is never reached and the test passes. If someone reorders the function later, the AssertionError surfaces instead of Denied and the test fails with a sentence describing what went wrong. The same trick applies anywhere an effect is reachable through a seam: swap the DB cursor for an object whose execute raises, swap the HTTP client for one whose send raises, and your deny tests start making a claim about sequencing rather than about exception types.

This is also the cheapest way to test an SSRF guard without network access in CI. You are not asserting that the request failed. You are asserting that no request was ever constructed.

Test the wiring, not just the guard

A guard can be correct, fully covered, and protect nothing, because the handler the MCP protocol actually dispatches to is a different callable than the one your unit tests import. An unguarded helper left registered, a decorator applied to the wrong function, a second code path added for a batch variant. Unit tests on resolve_in_sandbox cannot see any of that.

So add tests one layer up that go in by tool name, the way a client does, and assert the request is refused:

@pytest.mark.parametrize("tool,args", [
 ("read_text_file", {"path": "../../etc/passwd"}),
 ("query_database", {"sql": "DELETE FROM users"}),
 ("call_api", {"url": "http://169.254.169.254/latest/meta-data/"}),
])
def test_dispatch_refuses(tool, args):
 result = dispatch(tool, args)
 assert result.is_error

Guard-level tests tell you the decision is right. Dispatch-level tests tell you the decision is in the path. You want both, and they fail for different reasons, which is the point.

What this does not get you

Enumerated strings only cover the bypasses you thought of. A table of traversal inputs is a regression net, not a proof, and the interesting failures usually live one layer below your guard: how the OS normalizes a path before you ever see it, how symlinks and case-insensitive filesystems change what resolve() means, how a given SQL dialect treats a construct you did not plan for. If you need a stronger promise than string-level checking, you move the enforcement down to where the effect happens: a connection opened read-only, a filesystem handle confined by the kernel, an outbound proxy that only knows about hosts you listed.

Name-level checking has its own ceiling. Deciding that a hostname is allowed is a decision about a name, and the address behind a name is resolved separately. If you need an address-level guarantee, pin the resolved address and connect to that, or put the allowlist in a proxy outside the process.

And none of this is about resource use. A read-only guard says a statement cannot write. It says nothing about a statement that scans your largest table, so timeouts, row caps and statement limits are a separate job.

If your MCP server only exposes computation over data you already shipped inside it, skip most of this. The guards earn their keep the moment a tool takes a string from the model and turns it into a URL, a query, or a path.

I wrote these guards once and kept them. The MCP Starter Kit is the production-shaped Python MCP server I built, maintain and run, with the SSRF host allowlist, the read-only SQL guard, the path-traversal sandbox and a 14-test pytest suite over the guards and tool dispatch: https://fulcrumenterprises.tech/go/mcp-starter-kit/?c=devto&v=57dd17

What a timestamp proof actually proves, and the bytes you have to keep

Craig Solomon — Sun, 04 Oct 2026 13:00:00 +0000

Ask this about your own setup: if a timestamp proof verifies clean, what are you entitled to say out loud?

Here's the honest version. This exact digest existed no later than the block it was anchored in.

That's the whole claim. Not that the file is genuine. Not that you wrote it. Not that nobody has touched it since. A hash existed at a time, and nothing past that. Everything useful you can build on a timestamp proof rests on that sentence.

Start with the easy half, because you can run it right now:

pip install verify-proof
verify-proof hash contract.pdf

That gives you a SHA-256 digest of the file's bytes. The verify subcommand takes a proof file and the hash it's supposed to be for, recomputes that hash, and walks the Merkle path to the root itself instead of asking an API whether things look fine. Verification is local, no key and no account. The only subcommand that touches the network is create, which anchors a hash through ProofLedger and sends the hash, never the file.

So far so boring. Here's where it gets interesting.

The claim is attached to a digest, and the digest is attached to exact bytes

Once you accept that the proof is a statement about a digest, the next question is what you have to preserve for that statement to stay worth anything. The obvious answer is the proof file. Archive the proof, keep it somewhere durable, done.

That answer is not enough, and the gap is the point of this post.

A proof file is a pointer to a digest. The digest is a pointer to a specific sequence of bytes. If you still hold the proof but can no longer produce bytes that hash to that digest, you hold a verifiable statement about something you can't show anyone. The math still checks out. It just checks out about an artifact you lost.

And you can lose it without ever losing the file, which is the part worth internalizing. The digest is over bytes, not over content. Not over "the document", not over the text a human reads on screen. Change the container and leave the meaning identical, and the digest is gone.

Things that rewrite bytes while leaving content alone

I'm not going to tell you what's common in other people's pipelines, because I haven't read them. I'll tell you what to go look for in yours. Any step that deserializes and re-serializes is a candidate:

A PDF opened and re-saved by a different tool, which may rewrite the object table, the producer string, or embedded dates.
JSON parsed and dumped again, with different key order, different whitespace, or different unicode escaping.
Line ending conversion on checkout, or an editor normalizing the trailing newline.
Images re-encoded on upload, stripped of metadata, or thumbnailed in place.
A zip or tarball rebuilt, with different member order, different mtimes, different compression levels.
Office formats, which are zips, so everything above applies twice.

None of those are attacks. They're ordinary helpful behavior from ordinary tools. The proof doesn't know the difference between a helpful re-save and a malicious edit, and it shouldn't. That's not weakness, it's the thing you wanted. A digest that forgave cosmetic changes would also forgive the change somebody made on purpose.

Where this bites in practice

The failure doesn't show up when you create the proof. It shows up later, when something matters and you go to produce the artifact. By then the file has moved through a storage bucket, maybe a document viewer, maybe a mail client, maybe an export and a re-import. You still have a file called contract.pdf. It still looks right. It hashes to something else.

So the operational rule falls out of the mechanism rather than out of anybody's policy doc. Anchor the bytes you can commit to storing untouched, and store those bytes, not a convenient copy of them. If your workflow has to transform a file, decide which form is the one of record, hash that one, and let the transformed copies be copies. If you can't keep the raw bytes, anchoring a canonical serialization you can regenerate deterministically is a better bet than anchoring whatever came out of the tool that day.

The same thinking applies to anything you want to anchor from an agent session. There's an optional MCP server, pip install verify-proof[mcp], built on FastMCP, exposing five tools so Claude Desktop or Cursor can verify a proof or create one inside a conversation. Convenient, but it doesn't change the rule. Whatever bytes you handed the hash function are the bytes you're on the hook for.

The test, run it today

Pick a file you have a proof for, or any file you'd want one for. Then:

verify-proof hash it and write the digest down.
Push it through your normal path. Upload it and download it. Commit and check out on another machine. Attach it to a ticket and pull it back. Export and re-import. Whatever your real handling actually is.
Hash the result and compare.

If they match, you know your pipeline is byte-preserving and your proofs will survive it. If they don't, you just found out your archive copy and your anchored digest are about different things, which is much better to learn now than when you need the proof.

Second test, while you're in there. Take a proof you were issued by somebody and see whether you can check it with the issuer's service unreachable. If verifying requires their endpoint to answer, the proof is a claim with a dependency on the claimant staying online and willing. Proofs anchored to Polygon and to Bitcoin can be verified from the proof file and the chain, which includes legacy proofs issued by the retired ProofAnchor service, and that's exactly the case worth checking, because the issuer isn't there to ask anymore.

What I don't have a clean answer for

Formats you can't freeze. A record that lives in a database row, a doc that's edited collaboratively, anything where "the file" is a rendering rather than a stored artifact. You can anchor a canonical export, but then your proof is about your canonicalizer, and the canonicalizer becomes a thing you have to version and preserve alongside the proof. I've picked the raw bytes side of that tradeoff, and the cost is real: it pushes the discipline onto whoever stores the file.

If you've dealt with that for live records, I'd like to hear which way you went and what broke.

The tool is verify-proof, free and MIT licensed, published by Fulcrum Enterprises LLC. I built it, I maintain it, and I run it myself. Source is at https://github.com/Fulcrum-Enterprises/verify-proof and the package is on PyPI.

How an auto-publishing agent earns its permissions, channel by channel

Craig Solomon — Sat, 03 Oct 2026 13:00:00 +0000

A publish permission wants to be a boolean. AUTO_PUBLISH=true, ship it, go do something else.

The trouble with the boolean is what it encodes: that the agent is trustworthy in general. Trust is not general. A drafting agent can be perfectly sound writing a technical post for a dev feed and be a liability in a subreddit where a link in the body reads as spam. Same model, same prompt, different blast radius.

So in Content Agent Pro the auto-publish check is not a flag I set. It is a conjunction evaluated per (product, channel) pair, and most of the terms can go false without me touching the agent at all.

The gate

def can_autopublish(pair, cfg):
 if not cfg.master_switch:
 return False, "master auto-publish switch is off"
 if not pair.channel.free_api:
 return False, "channel has no free publishing API"
 if not pair.channel.credentials_present:
 return False, "credentials missing for this channel"

 reviewed = pair.approved + pair.rejected
 if reviewed < MIN_REVIEWED: # 10
 return False, f"only {reviewed} reviewed drafts"
 if pair.approved / reviewed < MIN_RATE: # 0.90
 return False, "approval rate below threshold"

 return True, "graduated"

MIN_REVIEWED = 10 and MIN_RATE = 0.90 are the graduation thresholds. Everything interesting about this function is in the details around those two numbers.

Evidence is reviewed drafts, not drafts

reviewed is approved + rejected. Pending drafts are not in the denominator, and they should not be. A draft nobody looked at is not evidence about anything. If I let total drafts drive the gate, a quiet week where the scheduler runs and I never open the dashboard would push a pair toward autonomy on the strength of work no human ever read. That is the exact inversion of what the gate is for.

This is worth checking in any track-record gate you build. The metric has to be judged outcomes, and the judgment has to have actually happened. Anything that accrues on its own is a clock, not a record.

The unit is the pair, not the agent

State lives per (product, channel). Product A on Dev.to can be graduated while product A on Reddit is still fully manual and product B on Dev.to has never been reviewed.

This falls out of how the drafting works. Each channel definition carries its own voice rules and link discipline, and link discipline in particular varies hard: footer link, profile-only, comment-only, no link at all. Nine channel definitions ship with the kit, and a draft that is honest and well-formed for one of them can be wrong for the next. The failure modes are not shared, so the trust record should not be either.

Practically, that means your schema key is a composite, and your approval counters are columns on that pair row rather than globals. If you find yourself writing SELECT avg(approved) FROM drafts, you have built a reputation score for the model. What you want is a reputation score for a specific job.

Two terms that are not about performance at all

master_switch and credentials_present are not evidence, and they are in the conjunction on purpose.

The master switch is the thing I can flip to make every graduated pair manual again, in one place, without editing per-pair state. If the only way to stop an agent is to walk through its permission records and revoke them one by one, you do not have a stop.

free_api is a constraint I encoded because of what the failure looks like when it is absent. Auto-publishing is only allowed on channels whose publishing API is free. Cost changes the shape of a runaway: a loop that posts too often on a free endpoint is an embarrassment you delete, and the same loop against a metered endpoint is a bill. The kit ships publishers for Dev.to and Bluesky, plus a Hashnode publisher that ships disabled, because Hashnode ended free API access and it now works only on a Hashnode Pro plan. That disabled publisher is the constraint doing its job in public rather than a feature I forgot to finish.

The rate only climbs if rejection carries information

A 90% approval threshold is only reachable if rejections change the next draft. So rejection in the dashboard is reject-with-reason, and the reason goes into a bounded window that gets injected into the next prompt for that pair: the last 8 reject reasons and the last 6 titles for the product and channel.

Both windows are bounded, for different reasons. Reject reasons go stale as the voice settles, and an unbounded list of them turns the prompt into an archive of complaints the model has already absorbed. Recent titles are there so the next draft does not restate the last one under a new headline.

If you are building this part yourself, the thing to get right is that the reason field is free text written at the moment of rejection. A dropdown of categories is easier to aggregate and far weaker as an input, because the useful part is the sentence, not the label.

A gate is not a substitute for a check

Here is the part that took a design decision rather than a threshold. A track record is a statement about the past. Approval rate cannot catch a claim the model invents today.

So the gate sits on top of deterministic validators that run on every draft, graduated or not. They are code, not prompt instructions. Any dollar amount, percentage, or large number that does not appear in the product's fact sheet hard-fails the draft. Hype words fail. AI-tell phrases fail. Unfilled placeholders fail. Per-product earnings language fails. Em dashes are handled differently: a sanitizer strips them, and the draft is not retried for it, because a character-level fix does not need a model round trip.

The division of labor is: validators define what is never allowed to publish, and the gate defines when a human stops having to look. Those are different questions, and a system that only has one of them either publishes fabrications with a clean track record or asks you to catch fabrications by eye forever.

In the dashboard, every draft card shows a validator flag count and the full findings, and /gates renders the graduation matrix so the state of each pair is visible rather than inferred.

Where this does not help

The honest limits, because they are the reason to decide deliberately rather than copy the pattern.

The threshold is not a guarantee. Ninety percent is not a hundred, and a graduated pair will eventually publish something you would have edited. The validators are what keeps that from being a fabricated number under your name. They do not keep it from being a mediocre post.

The record is historical and the model is not frozen. Change your product's fact sheet or your channel voice rules and the accumulated approval rate describes a job that no longer exists. Resetting a pair's counters after a material change to its inputs is a reasonable thing to add, and it is the kind of thing you have to decide for yourself, because only you know which config changes count as material.

Gates also assume you actually review. They convert human attention into autonomy. If the drafts pile up unread, nothing graduates, and correctly so.

And if you publish to one channel and you write the posts yourself, none of this is worth the schema. The mechanism earns its keep when there are several products, several channels with genuinely different rules, and a real cost to being wrong in public.

Content Agent Pro is the self-hosted, MIT-licensed version of this: a Python agent and a Next.js dashboard over one SQLite file, which I built, maintain, and run myself.

https://fulcrumenterprises.tech/go/content-agent-kit-pro/?c=devto

How to fold a Merkle path yourself, and why the root can't come from the proof file

Craig Solomon — Fri, 02 Oct 2026 13:00:00 +0000

There's one question worth asking about any timestamp proof file you're holding right now: when you verify it, what exactly are you comparing your computed root against, and where did that value come from?

Real question, real answer, and it's coming. But it doesn't mean much until you've folded a Merkle path by hand once, so let's do that part first.

What a proof file is asking you to do

The setup is simple. You want evidence that a file existed by a certain time. Hashing it gives you a digest. Writing that digest to a public chain gives you a timestamp. Writing one transaction per file gets expensive, so anchoring services batch: they take a pile of digests, build a Merkle tree over them, and publish the root.

Your digest is a leaf. The root is what actually got committed. The proof file is the receipt that connects the two, and it usually carries the leaf, the sibling hashes on the way up, the root, and something about where that root landed.

Verifying means doing the walk yourself. Hash your file, fold it upward with the siblings, and see whether you arrive at the root.

The leaf:

import hashlib

def leaf_hash(filename):
 with open(filename, "rb") as f:
 return hashlib.sha256(f.read()).digest()

The fold:

def fold(leaf, path):
 node = leaf
 for step in path:
 sibling = bytes.fromhex(step["hash"])
 if step["side"] == "left":
 node = hashlib.sha256(sibling + node).digest()
 else:
 node = hashlib.sha256(node + sibling).digest()
 return node.hex()

That's the mechanism. Your format will name those fields something else, and it may encode the side as an index instead of a word, but the shape is that: a loop, a concatenation, a digest.

The part that bites

Look at the branch in that loop. Order of concatenation is the entire content of the algorithm. sibling + node and node + sibling produce completely unrelated digests, and once you've gone the wrong way at one level, every level above it is garbage. The fold doesn't drift off by a little. It lands somewhere else entirely.

Which means there are a few things to pin down in whatever format you're handed, and they are not documented as loudly as you'd like:

Does the format tell you the side, or imply it? Reading an explicit side field and inferring side from index parity are different algorithms. So is sorting the pair before hashing, which some constructions do specifically to avoid carrying side information at all. Only one of those matches the tree that was actually built. Open your own proof file and find out which one it's describing.

Raw bytes or hex text? A digest has a binary form and a printable form, and hashing the printable form is a different operation from hashing the bytes it prints. If the tree was built by concatenating raw digests and your code concatenates hex strings, nothing will line up. The failure mode is nasty because it reads as "this proof is invalid" rather than "your decoder is wrong."

Once or twice? Some Merkle constructions hash each pair a single time, some apply SHA-256 twice at every internal node. Check which one your format means before you conclude anything about the file.

Every one of those is a place where a correct proof looks broken, or a broken proof looks fine because you and the builder disagree about what the bytes mean. This is why I'd rather have code that does the fold in front of me than a green check mark from somewhere else.

Where the obvious answer runs out

So you write the loop, you fold the path, you land exactly on the root in the file. Feels finished.

It isn't. Look at where each input came from.

The leaf came from your file, and that part is load bearing. It binds the proof to the actual bytes on your disk, so if one byte changes, the leaf changes and the fold stops reaching the root. Good.

But the path and the root both came out of the same document. So the thing you just proved is that the proof file is internally consistent with your file. That's a weaker statement than it feels like, because internal consistency is cheap to manufacture. Given any file at all, you can hash it, build a fresh tree that includes it, and emit a perfectly self-consistent proof today. The fold will check out. It says nothing about time, because nothing in it has been published anywhere.

The root is the only value in that file that was ever committed outside the file. That's the part that carries the timestamp. Everything else is derivation.

So the comparison that matters is not leaf-folded-against-root. It's root-against-chain.

What to look for in your own proof file

Open one. Find the root. Then ask whether the file gives you enough to resolve that root somewhere you don't control: a chain, a transaction identifier, a block. Something you could hand to a block explorer or your own node without asking the issuer's permission.

If the root only ever appears inside the file, and the only way to confirm it was ever anchored is to call the issuer's API and get back a boolean, then the trust boundary hasn't moved. You're back to taking their word for it, with extra hashing in the middle. Correct folding code doesn't save you from that, which is the uncomfortable bit. The crypto can be perfect and the chain of trust can still be a loop.

Where verify-proof sits, and what it won't do

verify-proof is a Python CLI, pip install verify-proof. Three subcommands: hash gives you the SHA-256 of a file, verify checks a proof file against that hash and its Merkle path, and create anchors a hash through ProofLedger, added in 0.3.0. It handles proofs anchored to Polygon and to Bitcoin, including legacy proofs from the retired ProofAnchor service. There's an optional MCP server, pip install verify-proof[mcp], built on FastMCP, exposing five tools, so you can verify or create inside a conversation in Claude Desktop or Cursor.

verify recomputes the hash and walks the path itself rather than asking an API whether the proof is good. Only create touches the network, and it sends the hash, never the file.

Now the limit, because it follows directly from that design decision. Since verification does no network calls, the tool cannot go look at the chain for you. It will do the fold and it will do it locally, on a machine with no route out, with no API key and no account. It will not tell you the root was ever anchored. That step stays yours.

I made that trade on purpose and I think it's the right one, but it is a trade, and I'd rather say so than describe an offline verifier as though it settles the on-chain question too.

The test you can run today

Take a proof file you already have, from whoever issued it.

Cover up the root. Pretend it isn't there.
Hash your file, then fold it up the path with the sibling hashes, using the loop above. Adjust for whatever your format says about side, encoding, and single or double hashing.
Uncover the root and compare.
Then take that computed root and go looking for it on the chain the file names, in an explorer or a node that is not the issuer's.

Read the result:

The fold doesn't reach the stated root. Either your reading of the format is off, or the file is. Fix the format questions first.
The fold reaches the root, but you can't find the root on chain. You have a self-consistent document. You don't have a timestamp.
The fold reaches the root, and the root is sitting in a block. Now you have something, and be precise about what: that digest existed by that block's time. Not that the file is genuine, not that anyone in particular wrote it. Just that those bytes existed by then.

And if step 4 turns into "call our endpoint and we'll confirm it," you've answered the question at the top of this post. The answer just isn't the one you wanted.

verify-proof is free and MIT licensed, published by Fulcrum Enterprises LLC. I built it and I maintain it. Source is at https://github.com/Fulcrum-Enterprises/verify-proof and the package is at pypi.org/project/verify-proof/, if you want something that does the fold in front of you.

Idempotent drafts: keeping a scheduled agent from writing the same post again

Craig Solomon — Thu, 01 Oct 2026 13:00:00 +0000

A scheduler is an at-least-once system wearing an exactly-once costume. You register a job, it fires on a clock, and for a while the mental model holds: the job runs when you said it would, the drafter calls a model, a row lands in the database, you go look at it.

Then reality shows up. The container restarts mid-run. You redeploy during a scheduled window. The host sleeps and the scheduler catches up on missed runs when it wakes. A model call times out at the transport layer, your retry wrapper fires, and the first call had already succeeded on the server side. The job body runs again, and because the job body is "draft something and insert it," you get another row.

Duplicate drafts are not a cosmetic problem. A drafting agent exists to feed a human review step, and review is the scarce resource. Every duplicate row is a decision a person has to make and then unmake. Worse, the duplicates are not identical: same schedule, same prompt, different sampling, so the reviewer cannot even skim and dismiss. They have to read both.

The fix is not a better scheduler. It is making the job body idempotent, so running it again is a no-op instead of a second draft.

The check-then-write trap

The obvious first move is to look before you leap:

def run(conn, platform, slot):
 existing = conn.execute(
 "SELECT id FROM drafts WHERE platform = ? AND slot = ?",
 (platform, slot),
 ).fetchone()
 if existing:
 return
 body = drafter.write(platform=platform, slot=slot)
 conn.execute(
 "INSERT INTO drafts (platform, slot, body, status) VALUES (?, ?, ?, 'ready')",
 (platform, slot, body),
 )
 conn.commit()

This is a time-of-check-to-time-of-use race with a very wide window in the middle, because drafter.write is a network call to a model. Anything that starts a second execution while the first one is waiting on that call passes the SELECT, because the insert has not happened yet. The check is honest and still wrong.

The general fix for this shape of bug is to stop asking the database a question and start giving it a constraint it can enforce.

Derive a key, let the database refuse

Give every draft a key that is a pure function of the work it represents, not of when the work happened:

def draft_key(platform: str, slot: str, recipe: str) -> str:
 return "|".join([platform, slot, recipe])

Three parts, and what goes in each one matters:

platform is the destination. The same idea drafted for different destinations is different work.
slot is the scheduled occurrence, derived from the run's scheduled time, not from the wall clock at execution. scheduled_for.strftime("%Y-%m-%dT%H") gives you a stable string that a catch-up run and the original run both produce. If you use now() here, a catch-up run gets its own key and you are back where you started.
recipe is a version marker for the drafter and prompt. This is the escape hatch: when you intentionally want a fresh draft for a slot you already drafted, you bump the recipe, and the new key is legitimately new.

Keep the key readable if you can. A key you can eyeball in a SQL shell is worth a lot during debugging. If the parts get long, hash them, but then log the plain parts next to the hash.

Then put the constraint where it cannot be raced:

CREATE TABLE IF NOT EXISTS drafts (
 id INTEGER PRIMARY KEY,
 key TEXT NOT NULL,
 platform TEXT NOT NULL,
 slot TEXT NOT NULL,
 status TEXT NOT NULL,
 body TEXT
);

CREATE UNIQUE INDEX IF NOT EXISTS drafts_key ON drafts (key);

If a dashboard reads the same database through Prisma, express the constraint there too so migrations carry it rather than living in a stray .sql file:

model Draft {
 id String @id @default(cuid())
 key String @unique
 platform String
 slot String
 status String
 body String?
}

Claim the slot before you spend the model call

A unique index alone turns the duplicate into a failed insert, which is correct but wasteful: you already paid for the model call before the database told you the work was redundant. Invert the order. Claim the key first, cheaply, then fill in the body.

def claim(conn, key, platform, slot):
 cur = conn.execute(
 "INSERT INTO drafts (key, platform, slot, status) "
 "VALUES (?, ?, ?, 'claimed') "
 "ON CONFLICT(key) DO NOTHING",
 (key, platform, slot),
 )
 conn.commit()
 return bool(cur.rowcount)


def run(conn, platform, slot, recipe):
 key = draft_key(platform, slot, recipe)
 if not claim(conn, key, platform, slot):
 log.info("slot already claimed, skipping: %s", key)
 return
 body = drafter.write(platform=platform, slot=slot)
 conn.execute(
 "UPDATE drafts SET body = ?, status = 'ready' WHERE key = ?",
 (body, key),
 )
 conn.commit()

ON CONFLICT DO NOTHING plus rowcount is the whole trick. The insert either wins or it does not, decided inside the database, and the loser never calls the model. There is no window to race because the claim is one statement.

Two SQLite details make this behave under concurrency. Turn on WAL so a reader (a dashboard rendering the review queue) does not block the writer: PRAGMA journal_mode=WAL. And set a busy timeout on every connection, so a writer that arrives during another write waits for the lock instead of raising immediately. Without the timeout, a claim that collides with an unrelated write surfaces as a locked-database error, and you will misread it as a bug in the claim logic.

Keep writes short, too. Claim, release, do the slow network work outside any transaction, then reopen for the update. Holding a write transaction open across a model call is the fastest way to make a single-file database feel broken.

Where this approach stops helping

It catches exact re-execution of the same unit of work. It does not deduplicate meaning. Bump the recipe and you get a new key and a genuinely new draft, even if the model produces text that reads almost the same as what is already sitting in review. Semantic near-duplicates are a different problem and a unique index will never see them.

It leaves a half-finished state you have to handle. If the process dies between the claim and the update, you keep a claimed row with a null body, and the key is now taken, so retries skip it forever. You need a sweeper that finds claimed rows with no body older than a cutoff you choose and either resets them to unclaimed or deletes them. Pick that cutoff to be comfortably longer than your slowest model call, otherwise the sweeper starts stealing slots from runs that are still working.

It does nothing about ordering or about throughput. Idempotency is not a queue. If you need retries with backoff, priorities, or workers on separate machines, you want a real job store, and at that point SQLite is not the right shared surface.

And it does not solve the actual hard part of a content agent, which is the approval step. A clean, deduplicated queue that nobody reviews is still a pile of unpublished text. The value of the key is that it protects the reviewer's attention, which only matters if there is a place to review.

If you want that half already running instead of assembled from scratch, I build and maintain the AI Content Agent Kit: a Python drafting agent and a Next.js approval dashboard in one docker-compose, MIT licensed.

https://fulcrumenterprises.tech/go/content-agent-kit/?c=devto

stdout is the protocol: what changes when an MCP server moves from stdio to HTTP

Craig Solomon — Wed, 30 Sep 2026 13:00:00 +0000

On stdio transport, an MCP client starts your server as a child process and talks to it over stdin and stdout. That is the whole channel. Stdout is not "where your program prints things while the real protocol happens somewhere else." Stdout is the wire.

Which means this tool is broken:

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("demo")

@mcp.tool()
def list_tables() -> list[str]:
 print("listing tables") # this goes onto the JSON-RPC stream
 return ["users", "orders"]

The function returns the right value. The client still fails, because listing tables\n arrived in the middle of a message stream that only expects framed JSON-RPC. Depending on the client you get a parse error, a tool that never appears, or a server that disconnects during startup with nothing useful on screen.

The failure is confusing because the cause and the symptom are far apart. You added a debug line to a tool and the whole server stopped loading in Claude Desktop.

The fix is one line, and it is about handlers, not about print

Send everything human-readable to stderr:

import sys
import logging

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(logging.Formatter("%(levelname)s %(name)s %(message)s"))
logging.basicConfig(level=logging.INFO, handlers=[handler])

log = logging.getLogger("demo")

@mcp.tool()
def list_tables() -> list[str]:
 log.info("list_tables called")
 return ["users", "orders"]

logging.basicConfig() already defaults to stderr, so you may think you get this for free. You do not, reliably. Whoever calls basicConfig first wins, and if an imported module configures the root logger with stream=sys.stdout, your log lines land on the protocol channel. Configure the handler explicitly in your entrypoint, before you import anything chatty.

The other source of stray bytes is import-time output from a dependency: a banner, a deprecation notice someone routed through print, a progress bar. For Python-level writes you can fence the import:

import sys
import contextlib

with contextlib.redirect_stdout(sys.stderr):
 import chatty_dependency

redirect_stdout swaps sys.stdout for the duration of the block. It does not touch file descriptor 1, so a C extension that writes to fd 1 directly walks straight past it. If you hit that, you are into os.dup2 territory, and at that point it is usually easier to debug over HTTP and keep the dependency out of the stdio path.

A useful habit while writing tools: treat print as a syntax error in server code. Grep for it before you ship.

The switch itself is boring, which is the point

Both transports run the same tool functions. Picking one is an environment decision, not a code decision:

import os

if __name__ == "__main__":
 transport = os.environ.get("MCP_TRANSPORT", "stdio")
 if transport == "streamable-http":
 mcp.run(transport="streamable-http")
 else:
 mcp.run(transport="stdio")

Name the variable whatever you like. The value of doing it this way is that your local Claude Desktop setup and your deployed container are the same artifact, so a tool you debugged locally is the tool that runs remotely.

What is not boring is everything the switch implies.

What changes when you stop being a subprocess

The trust boundary moves. On stdio, your server is a child process of the client, on the user's machine, running with the user's privileges. Nobody else can reach it, because there is nothing to reach. The only thing that drives it is the model sitting in front of one user. On streamable-http, you have a listening socket. Now the interesting question is who can open a connection to it, and the answer is no longer "only the person who launched it."

That is the moment the input guards stop being a nicety. A file tool that resolves paths inside a sandbox root, a SQL tool that rejects anything but reads, an HTTP tool that checks the destination host against an allowlist: on stdio those protect a user from their own model. Over HTTP they protect you from anyone who can reach the endpoint.

Process state becomes shared state. One stdio process serves one client. One HTTP process serves many sessions at once. Module-level globals that felt harmless locally stop being harmless:

# fine as a subprocess, a bug as a server
CURRENT_DIR = None

@mcp.tool()
def set_dir(path: str) -> str:
 global CURRENT_DIR
 CURRENT_DIR = path
 return "ok"

Two callers, one variable. Write tools stateless: take the path, the query, the URL as an argument on every call, and derive everything else from it. If you need a cached resource like a database connection, make it per-request or make it thread-safe on purpose rather than by accident.

Blocking calls start costing other people. Under stdio, a synchronous call that blocks for a while only makes one user wait. Under HTTP, it can stall concurrent requests. Use an async HTTP client for outbound calls, and push blocking work such as a synchronous database driver into a thread with asyncio.to_thread instead of calling it directly inside an async handler.

Debug without the transport at all

The best answer to "I cannot print, so how do I see what is happening" is to stop debugging through the client. Your tool functions are ordinary Python. Call them directly:

import pytest

def test_read_text_file_rejects_escape():
 with pytest.raises(ValueError):
 read_text_file("../../etc/passwd")

No transport, no client, no handshake, and the denial cases are the ones you actually want pinned down. When something misbehaves through Claude Desktop, checking whether the same call misbehaves in a test tells you immediately whether you have a tool bug or a transport bug. Those have completely different fixes and it is easy to spend an afternoon on the wrong one.

Limits

Routing logs to stderr is hygiene, not a feature. It stops a specific, repeatable crash. It does nothing for a tool that returns the wrong data.

An env var that switches transports does not give you authentication, TLS, or rate limiting. streamable-http makes the server reachable. Deciding who may reach it is a deployment question your mcp.run call does not answer, and if you expose an endpoint on the open internet with no auth in front of it, per-call input guards are not going to save you.

Those guards are also input validation, not authorization. They answer "is this call shaped like something allowed" and never "is this caller allowed to see this row." If different callers must see different data, that belongs in a layer that knows who the caller is.

And if you only ever run inside Claude Desktop on your own laptop, the HTTP path may be work you do not need. The reason to wire it early is that retrofitting statelessness into tools written against a single-process assumption is more annoying than writing them stateless on day one.

The MCP Starter Kit is the Python MCP server I built and maintain, with both transports behind one env var, the SSRF, SQL and path guards, and a pytest suite already in place: https://fulcrumenterprises.tech/go/mcp-starter-kit/?c=devto