DEV Community: Craig Solomon

What crosses the network when you timestamp a file, and how to check it yourself

Craig Solomon — Tue, 06 Oct 2026 13:00:00 +0000

Here's the question worth asking about your own provenance setup before any of the fun parts: at verification time, what has to be up?

Not at build time. At verification time. Months from now, when someone asks you to prove a file existed on the date you say it did, which machines, which services, which credentials have to be alive and reachable for you to answer?

It's an easy question to skip, because the issuing side is the part you actually test. You wire up the attestation step, it goes green in CI, you move on.

So let's work it through properly. There are two phases and they have completely different dependency profiles.

Phase one: issuing has to touch the network, and that's fine

If you want a digest anchored to a chain, something has to put it there. There's no clever way around that. You hash the file locally and then you hand that hash to a service that batches it into a Merkle tree and anchors the root.

In my tool that's one subcommand:

pip install verify-proof

verify-proof hash ... # SHA-256 a file
verify-proof verify ... # check a proof against the hash and its Merkle path
verify-proof create ... # anchor a hash (added in 0.3.0)

Three subcommands. Exactly one of them opens a socket: create. It needs a free ProofLedger API key, and it sends the hash, never the file.

Now, the obvious reading of that last sentence is "good, nothing leaks, we're done on the egress question."

That reading is not enough, and this is the part of the post I actually care about.

A digest is a commitment, not a blindfold

Sending a hash instead of a file is not the same as sending nothing. A SHA-256 digest is a commitment to the content. Anyone holding it can't read your file, but they can confirm a guess at it. Feed in a candidate, hash it, compare.

Whether that matters depends entirely on how guessable the content is. A multi-megabyte binary with embedded build IDs, fine, nobody is guessing that. A one-line config value. A short contract built from a template where the only variable fields are a name and a number. A CSV row. Those are a different story, and the hash of one of them is closer to a lookup key than to a secret.

So the protection in a hash-only transport is not "a hash is unreadable." It's "the file never moved off the machine." That's a real property and it's worth stating in those terms, because stated correctly you can see what it does and does not buy you, and you can decide for yourself whether the digest of this particular artifact is something you're comfortable handing to anyone.

If it isn't, salt the thing before you hash it. Hash a file that includes a random value you keep alongside the proof. Then the digest is no longer a lookup key for a guessable document, and you've traded it for one more piece of state you have to not lose. That's the tradeoff, and it's yours to make rather than mine.

Phase two: verifying, and what "local" actually has to mean

Here's where the dependency question gets sharp.

Verification is hashing and more hashing. Recompute SHA-256 over the file, then fold your leaf up the Merkle path, concatenating and hashing at each level until you have a root. That's all arithmetic. No network required.

Except the root you just computed is worthless on its own. You have to compare it against the root that's actually anchored on chain, and that value cannot come from the proof file, because the proof file is the thing you're checking. So it comes from a block explorer, from a node you run, or from a copy of the root you captured and trusted at the time.

Which means "offline" was never the right property to ask for. The right property is narrower and more useful: verification must not depend on the issuer. A chain lookup is a dependency you can satisfy many ways, including with hardware you control. A call to api.vendor.com/verify is a dependency with exactly one supplier, and that supplier is the party whose claim you're trying to check.

That's the whole reason I wrote the verify path to recompute and fold locally rather than ask an API whether the proof is good. It needs no key and no account. The asymmetry is deliberate: creating a proof costs you a credential and a network call, checking one costs you neither. That also means it can still check proofs issued by ProofAnchor, a service that's retired and isn't coming back. The anchors are on Polygon and Bitcoin, and both of those will answer a question about an old block whether or not the issuer still exists.

Same split holds if you run it as an MCP server, which is optional (pip install verify-proof[mcp], built on FastMCP) and exposes five tools so Claude Desktop or Cursor can do this inside a conversation. The create tool still wants a key and still talks out. The verify side still doesn't.

The test, which you can run today against whatever you're using now

Forget my tool for a second. Here's how to find out what your current setup depends on. Takes one sitting.

Kill the issuer, keep the chain. Block egress to your vendor's API hostname at the firewall and leave the rest of your network up. Then verify a proof you already hold. If it passes, your verify path is talking to a chain and doing its own math. If it fails, you've learned that your ability to substantiate your own claim is a function of that company's uptime and your account's standing.

Pull the credential. Unset every API key and token in the environment and verify again. A verifier that needs a credential is a verifier that can be revoked.

Watch the issuing call. Compute the digest yourself first with sha256sum, then capture the traffic while the attestation step runs and search the capture:

sha256sum contract.pdf # note the hex digest

# run your create/attest step with a capture or proxy in front of it, then:
grep -c '<that digest>' capture.txt # expect a hit
grep -c '<a string that exists only in the file>' capture.txt # expect zero

If the second grep returns anything above zero, your file body is going over the wire and you should know that before the next audit, not during it.

Check what the root is compared against. Read your proof file. If the anchored root is sitting in it and your verifier compares the computed root to that field, the check is internally consistent and externally meaningless.

One honest limit on all of this: none of it tells you a file is genuine, or that a particular person wrote it. It tells you a specific sequence of bytes existed at a specific time. That's narrow, and it's often exactly the thing in dispute, but don't let it grow in the retelling.

verify-proof is MIT licensed and free, on PyPI and at https://github.com/Fulcrum-Enterprises/verify-proof. I built it and I run it.

The part I'd genuinely like other people's answers on: for long-lived artifacts, do you store the anchored root yourself at issue time, or do you plan on querying the chain whenever the question comes up? I went with the second and I keep wondering if the first is the adult decision.

Disclosure: this article was drafted by an AI agent I built and run, from facts I supplied about my own project.

Testing the deny path: how to prove an MCP server's guards hold

Craig Solomon — Mon, 05 Oct 2026 13:00:00 +0000

An MCP server's guards are the only code in the project where the test that matters asserts that nothing happened. No row came back. No request left the box. No file was opened. That is an awkward thing to assert, and it is the reason a guard suite can be green while proving almost nothing about the guard.

Here is the shape I use, and the two mistakes it is designed to catch.

Make the guard a decision you can call

If the check lives inside the tool body, the only way to test it is to drive the whole tool and infer the decision from whatever came back. Pull the decision out so it is a function with a return value or an exception, and nothing else.

# tools/files.py
from pathlib import Path


class Denied(Exception):
 pass


def resolve_in_sandbox(root: Path, requested: str) -> Path:
 candidate = (root / requested).resolve()
 if not candidate.is_relative_to(root.resolve()):
 raise Denied(f"outside sandbox: {requested}")
 return candidate


def read_text_file(root: Path, requested: str) -> str:
 return resolve_in_sandbox(root, requested).read_text(encoding="utf-8")

The same split works for the other two reaches an MCP server tends to hand a model. A SQL guard is assert_read_only(sql) raising before execute. An SSRF guard is assert_host_allowed(url) raising before the HTTP client is ever handed the URL. In every case the guard is a pure decision over an input string, and the effect is a separate line that runs afterwards.

Now the deny cases are a table:

import pytest
from pathlib import Path
from tools.files import Denied, read_text_file, resolve_in_sandbox


@pytest.fixture
def root(tmp_path):
 (tmp_path / "notes.txt").write_text("ok")
 return tmp_path


@pytest.mark.parametrize("requested", [
 "../../etc/passwd",
 "/etc/passwd",
 "notes.txt/../../../etc/passwd",
 "./../notes.txt",
])
def test_escape_is_denied(root, requested):
 with pytest.raises(Denied):
 resolve_in_sandbox(root, requested)

Parametrize rather than loop inside one test. When a new bypass turns up you add a string, and when one regresses pytest names the exact input in the failure line instead of telling you that test_paths failed.

The deny-only suite that passes for the wrong reason

Here is mistake one. A guard test file made entirely of pytest.raises blocks passes if you replace the body of the guard with raise Denied("nope"). Every attack string is rejected. So is every legitimate request. The suite is green and the tool is dead.

So pin the allow path in the same file, right next to the denials:

def test_plain_name_is_allowed(root):
 assert resolve_in_sandbox(root, "notes.txt").read_text() == "ok"

You can check this property deliberately, and it costs one edit. Make the guard unconditionally deny, run the suite, and look at what fails. If only your allow tests go red, the deny tests are real. If nothing goes red, your deny tests were never testing the guard. Then make the guard unconditionally allow and confirm the whole deny table lights up. Undo both. That pair of edits tells you more about a guard suite than reading it does.

Asserting that nothing happened

Mistake two is subtler. pytest.raises(Denied) proves an exception came out. It does not prove the effect never ran. A guard that checks the path after opening the file raises exactly the same exception and leaks exactly the same bytes into your process first. Order is the whole property, and the exception type cannot see order.

You can test order directly by booby-trapping the effect:

def test_denied_read_never_touches_disk(root, monkeypatch):
 def explode(*args, **kwargs):
 raise AssertionError("read_text ran on a denied path")

 monkeypatch.setattr(Path, "read_text", explode)

 with pytest.raises(Denied):
 read_text_file(root, "../../etc/passwd")

If the guard runs first, read_text is never reached and the test passes. If someone reorders the function later, the AssertionError surfaces instead of Denied and the test fails with a sentence describing what went wrong. The same trick applies anywhere an effect is reachable through a seam: swap the DB cursor for an object whose execute raises, swap the HTTP client for one whose send raises, and your deny tests start making a claim about sequencing rather than about exception types.

This is also the cheapest way to test an SSRF guard without network access in CI. You are not asserting that the request failed. You are asserting that no request was ever constructed.

Test the wiring, not just the guard

A guard can be correct, fully covered, and protect nothing, because the handler the MCP protocol actually dispatches to is a different callable than the one your unit tests import. An unguarded helper left registered, a decorator applied to the wrong function, a second code path added for a batch variant. Unit tests on resolve_in_sandbox cannot see any of that.

So add tests one layer up that go in by tool name, the way a client does, and assert the request is refused:

@pytest.mark.parametrize("tool,args", [
 ("read_text_file", {"path": "../../etc/passwd"}),
 ("query_database", {"sql": "DELETE FROM users"}),
 ("call_api", {"url": "http://169.254.169.254/latest/meta-data/"}),
])
def test_dispatch_refuses(tool, args):
 result = dispatch(tool, args)
 assert result.is_error

Guard-level tests tell you the decision is right. Dispatch-level tests tell you the decision is in the path. You want both, and they fail for different reasons, which is the point.

What this does not get you

Enumerated strings only cover the bypasses you thought of. A table of traversal inputs is a regression net, not a proof, and the interesting failures usually live one layer below your guard: how the OS normalizes a path before you ever see it, how symlinks and case-insensitive filesystems change what resolve() means, how a given SQL dialect treats a construct you did not plan for. If you need a stronger promise than string-level checking, you move the enforcement down to where the effect happens: a connection opened read-only, a filesystem handle confined by the kernel, an outbound proxy that only knows about hosts you listed.

Name-level checking has its own ceiling. Deciding that a hostname is allowed is a decision about a name, and the address behind a name is resolved separately. If you need an address-level guarantee, pin the resolved address and connect to that, or put the allowlist in a proxy outside the process.

And none of this is about resource use. A read-only guard says a statement cannot write. It says nothing about a statement that scans your largest table, so timeouts, row caps and statement limits are a separate job.

If your MCP server only exposes computation over data you already shipped inside it, skip most of this. The guards earn their keep the moment a tool takes a string from the model and turns it into a URL, a query, or a path.

I wrote these guards once and kept them. The MCP Starter Kit is the production-shaped Python MCP server I built, maintain and run, with the SSRF host allowlist, the read-only SQL guard, the path-traversal sandbox and a 14-test pytest suite over the guards and tool dispatch: https://fulcrumenterprises.tech/go/mcp-starter-kit/?c=devto&v=57dd17

What a timestamp proof actually proves, and the bytes you have to keep

Craig Solomon — Sun, 04 Oct 2026 13:00:00 +0000

Ask this about your own setup: if a timestamp proof verifies clean, what are you entitled to say out loud?

Here's the honest version. This exact digest existed no later than the block it was anchored in.

That's the whole claim. Not that the file is genuine. Not that you wrote it. Not that nobody has touched it since. A hash existed at a time, and nothing past that. Everything useful you can build on a timestamp proof rests on that sentence.

Start with the easy half, because you can run it right now:

pip install verify-proof
verify-proof hash contract.pdf

That gives you a SHA-256 digest of the file's bytes. The verify subcommand takes a proof file and the hash it's supposed to be for, recomputes that hash, and walks the Merkle path to the root itself instead of asking an API whether things look fine. Verification is local, no key and no account. The only subcommand that touches the network is create, which anchors a hash through ProofLedger and sends the hash, never the file.

So far so boring. Here's where it gets interesting.

The claim is attached to a digest, and the digest is attached to exact bytes

Once you accept that the proof is a statement about a digest, the next question is what you have to preserve for that statement to stay worth anything. The obvious answer is the proof file. Archive the proof, keep it somewhere durable, done.

That answer is not enough, and the gap is the point of this post.

A proof file is a pointer to a digest. The digest is a pointer to a specific sequence of bytes. If you still hold the proof but can no longer produce bytes that hash to that digest, you hold a verifiable statement about something you can't show anyone. The math still checks out. It just checks out about an artifact you lost.

And you can lose it without ever losing the file, which is the part worth internalizing. The digest is over bytes, not over content. Not over "the document", not over the text a human reads on screen. Change the container and leave the meaning identical, and the digest is gone.

Things that rewrite bytes while leaving content alone

I'm not going to tell you what's common in other people's pipelines, because I haven't read them. I'll tell you what to go look for in yours. Any step that deserializes and re-serializes is a candidate:

A PDF opened and re-saved by a different tool, which may rewrite the object table, the producer string, or embedded dates.
JSON parsed and dumped again, with different key order, different whitespace, or different unicode escaping.
Line ending conversion on checkout, or an editor normalizing the trailing newline.
Images re-encoded on upload, stripped of metadata, or thumbnailed in place.
A zip or tarball rebuilt, with different member order, different mtimes, different compression levels.
Office formats, which are zips, so everything above applies twice.

None of those are attacks. They're ordinary helpful behavior from ordinary tools. The proof doesn't know the difference between a helpful re-save and a malicious edit, and it shouldn't. That's not weakness, it's the thing you wanted. A digest that forgave cosmetic changes would also forgive the change somebody made on purpose.

Where this bites in practice

The failure doesn't show up when you create the proof. It shows up later, when something matters and you go to produce the artifact. By then the file has moved through a storage bucket, maybe a document viewer, maybe a mail client, maybe an export and a re-import. You still have a file called contract.pdf. It still looks right. It hashes to something else.

So the operational rule falls out of the mechanism rather than out of anybody's policy doc. Anchor the bytes you can commit to storing untouched, and store those bytes, not a convenient copy of them. If your workflow has to transform a file, decide which form is the one of record, hash that one, and let the transformed copies be copies. If you can't keep the raw bytes, anchoring a canonical serialization you can regenerate deterministically is a better bet than anchoring whatever came out of the tool that day.

The same thinking applies to anything you want to anchor from an agent session. There's an optional MCP server, pip install verify-proof[mcp], built on FastMCP, exposing five tools so Claude Desktop or Cursor can verify a proof or create one inside a conversation. Convenient, but it doesn't change the rule. Whatever bytes you handed the hash function are the bytes you're on the hook for.

The test, run it today

Pick a file you have a proof for, or any file you'd want one for. Then:

verify-proof hash it and write the digest down.
Push it through your normal path. Upload it and download it. Commit and check out on another machine. Attach it to a ticket and pull it back. Export and re-import. Whatever your real handling actually is.
Hash the result and compare.

If they match, you know your pipeline is byte-preserving and your proofs will survive it. If they don't, you just found out your archive copy and your anchored digest are about different things, which is much better to learn now than when you need the proof.

Second test, while you're in there. Take a proof you were issued by somebody and see whether you can check it with the issuer's service unreachable. If verifying requires their endpoint to answer, the proof is a claim with a dependency on the claimant staying online and willing. Proofs anchored to Polygon and to Bitcoin can be verified from the proof file and the chain, which includes legacy proofs issued by the retired ProofAnchor service, and that's exactly the case worth checking, because the issuer isn't there to ask anymore.

What I don't have a clean answer for

Formats you can't freeze. A record that lives in a database row, a doc that's edited collaboratively, anything where "the file" is a rendering rather than a stored artifact. You can anchor a canonical export, but then your proof is about your canonicalizer, and the canonicalizer becomes a thing you have to version and preserve alongside the proof. I've picked the raw bytes side of that tradeoff, and the cost is real: it pushes the discipline onto whoever stores the file.

If you've dealt with that for live records, I'd like to hear which way you went and what broke.

The tool is verify-proof, free and MIT licensed, published by Fulcrum Enterprises LLC. I built it, I maintain it, and I run it myself. Source is at https://github.com/Fulcrum-Enterprises/verify-proof and the package is on PyPI.

How an auto-publishing agent earns its permissions, channel by channel

Craig Solomon — Sat, 03 Oct 2026 13:00:00 +0000

A publish permission wants to be a boolean. AUTO_PUBLISH=true, ship it, go do something else.

The trouble with the boolean is what it encodes: that the agent is trustworthy in general. Trust is not general. A drafting agent can be perfectly sound writing a technical post for a dev feed and be a liability in a subreddit where a link in the body reads as spam. Same model, same prompt, different blast radius.

So in Content Agent Pro the auto-publish check is not a flag I set. It is a conjunction evaluated per (product, channel) pair, and most of the terms can go false without me touching the agent at all.

The gate

def can_autopublish(pair, cfg):
 if not cfg.master_switch:
 return False, "master auto-publish switch is off"
 if not pair.channel.free_api:
 return False, "channel has no free publishing API"
 if not pair.channel.credentials_present:
 return False, "credentials missing for this channel"

 reviewed = pair.approved + pair.rejected
 if reviewed < MIN_REVIEWED: # 10
 return False, f"only {reviewed} reviewed drafts"
 if pair.approved / reviewed < MIN_RATE: # 0.90
 return False, "approval rate below threshold"

 return True, "graduated"

MIN_REVIEWED = 10 and MIN_RATE = 0.90 are the graduation thresholds. Everything interesting about this function is in the details around those two numbers.

Evidence is reviewed drafts, not drafts

reviewed is approved + rejected. Pending drafts are not in the denominator, and they should not be. A draft nobody looked at is not evidence about anything. If I let total drafts drive the gate, a quiet week where the scheduler runs and I never open the dashboard would push a pair toward autonomy on the strength of work no human ever read. That is the exact inversion of what the gate is for.

This is worth checking in any track-record gate you build. The metric has to be judged outcomes, and the judgment has to have actually happened. Anything that accrues on its own is a clock, not a record.

The unit is the pair, not the agent

State lives per (product, channel). Product A on Dev.to can be graduated while product A on Reddit is still fully manual and product B on Dev.to has never been reviewed.

This falls out of how the drafting works. Each channel definition carries its own voice rules and link discipline, and link discipline in particular varies hard: footer link, profile-only, comment-only, no link at all. Nine channel definitions ship with the kit, and a draft that is honest and well-formed for one of them can be wrong for the next. The failure modes are not shared, so the trust record should not be either.

Practically, that means your schema key is a composite, and your approval counters are columns on that pair row rather than globals. If you find yourself writing SELECT avg(approved) FROM drafts, you have built a reputation score for the model. What you want is a reputation score for a specific job.

Two terms that are not about performance at all

master_switch and credentials_present are not evidence, and they are in the conjunction on purpose.

The master switch is the thing I can flip to make every graduated pair manual again, in one place, without editing per-pair state. If the only way to stop an agent is to walk through its permission records and revoke them one by one, you do not have a stop.

free_api is a constraint I encoded because of what the failure looks like when it is absent. Auto-publishing is only allowed on channels whose publishing API is free. Cost changes the shape of a runaway: a loop that posts too often on a free endpoint is an embarrassment you delete, and the same loop against a metered endpoint is a bill. The kit ships publishers for Dev.to and Bluesky, plus a Hashnode publisher that ships disabled, because Hashnode ended free API access and it now works only on a Hashnode Pro plan. That disabled publisher is the constraint doing its job in public rather than a feature I forgot to finish.

The rate only climbs if rejection carries information

A 90% approval threshold is only reachable if rejections change the next draft. So rejection in the dashboard is reject-with-reason, and the reason goes into a bounded window that gets injected into the next prompt for that pair: the last 8 reject reasons and the last 6 titles for the product and channel.

Both windows are bounded, for different reasons. Reject reasons go stale as the voice settles, and an unbounded list of them turns the prompt into an archive of complaints the model has already absorbed. Recent titles are there so the next draft does not restate the last one under a new headline.

If you are building this part yourself, the thing to get right is that the reason field is free text written at the moment of rejection. A dropdown of categories is easier to aggregate and far weaker as an input, because the useful part is the sentence, not the label.

A gate is not a substitute for a check

Here is the part that took a design decision rather than a threshold. A track record is a statement about the past. Approval rate cannot catch a claim the model invents today.

So the gate sits on top of deterministic validators that run on every draft, graduated or not. They are code, not prompt instructions. Any dollar amount, percentage, or large number that does not appear in the product's fact sheet hard-fails the draft. Hype words fail. AI-tell phrases fail. Unfilled placeholders fail. Per-product earnings language fails. Em dashes are handled differently: a sanitizer strips them, and the draft is not retried for it, because a character-level fix does not need a model round trip.

The division of labor is: validators define what is never allowed to publish, and the gate defines when a human stops having to look. Those are different questions, and a system that only has one of them either publishes fabrications with a clean track record or asks you to catch fabrications by eye forever.

In the dashboard, every draft card shows a validator flag count and the full findings, and /gates renders the graduation matrix so the state of each pair is visible rather than inferred.

Where this does not help

The honest limits, because they are the reason to decide deliberately rather than copy the pattern.

The threshold is not a guarantee. Ninety percent is not a hundred, and a graduated pair will eventually publish something you would have edited. The validators are what keeps that from being a fabricated number under your name. They do not keep it from being a mediocre post.

The record is historical and the model is not frozen. Change your product's fact sheet or your channel voice rules and the accumulated approval rate describes a job that no longer exists. Resetting a pair's counters after a material change to its inputs is a reasonable thing to add, and it is the kind of thing you have to decide for yourself, because only you know which config changes count as material.

Gates also assume you actually review. They convert human attention into autonomy. If the drafts pile up unread, nothing graduates, and correctly so.

And if you publish to one channel and you write the posts yourself, none of this is worth the schema. The mechanism earns its keep when there are several products, several channels with genuinely different rules, and a real cost to being wrong in public.

Content Agent Pro is the self-hosted, MIT-licensed version of this: a Python agent and a Next.js dashboard over one SQLite file, which I built, maintain, and run myself.

https://fulcrumenterprises.tech/go/content-agent-kit-pro/?c=devto

How to fold a Merkle path yourself, and why the root can't come from the proof file

Craig Solomon — Fri, 02 Oct 2026 13:00:00 +0000

There's one question worth asking about any timestamp proof file you're holding right now: when you verify it, what exactly are you comparing your computed root against, and where did that value come from?

Real question, real answer, and it's coming. But it doesn't mean much until you've folded a Merkle path by hand once, so let's do that part first.

What a proof file is asking you to do

The setup is simple. You want evidence that a file existed by a certain time. Hashing it gives you a digest. Writing that digest to a public chain gives you a timestamp. Writing one transaction per file gets expensive, so anchoring services batch: they take a pile of digests, build a Merkle tree over them, and publish the root.

Your digest is a leaf. The root is what actually got committed. The proof file is the receipt that connects the two, and it usually carries the leaf, the sibling hashes on the way up, the root, and something about where that root landed.

Verifying means doing the walk yourself. Hash your file, fold it upward with the siblings, and see whether you arrive at the root.

The leaf:

import hashlib

def leaf_hash(filename):
 with open(filename, "rb") as f:
 return hashlib.sha256(f.read()).digest()

The fold:

def fold(leaf, path):
 node = leaf
 for step in path:
 sibling = bytes.fromhex(step["hash"])
 if step["side"] == "left":
 node = hashlib.sha256(sibling + node).digest()
 else:
 node = hashlib.sha256(node + sibling).digest()
 return node.hex()

That's the mechanism. Your format will name those fields something else, and it may encode the side as an index instead of a word, but the shape is that: a loop, a concatenation, a digest.

The part that bites

Look at the branch in that loop. Order of concatenation is the entire content of the algorithm. sibling + node and node + sibling produce completely unrelated digests, and once you've gone the wrong way at one level, every level above it is garbage. The fold doesn't drift off by a little. It lands somewhere else entirely.

Which means there are a few things to pin down in whatever format you're handed, and they are not documented as loudly as you'd like:

Does the format tell you the side, or imply it? Reading an explicit side field and inferring side from index parity are different algorithms. So is sorting the pair before hashing, which some constructions do specifically to avoid carrying side information at all. Only one of those matches the tree that was actually built. Open your own proof file and find out which one it's describing.

Raw bytes or hex text? A digest has a binary form and a printable form, and hashing the printable form is a different operation from hashing the bytes it prints. If the tree was built by concatenating raw digests and your code concatenates hex strings, nothing will line up. The failure mode is nasty because it reads as "this proof is invalid" rather than "your decoder is wrong."

Once or twice? Some Merkle constructions hash each pair a single time, some apply SHA-256 twice at every internal node. Check which one your format means before you conclude anything about the file.

Every one of those is a place where a correct proof looks broken, or a broken proof looks fine because you and the builder disagree about what the bytes mean. This is why I'd rather have code that does the fold in front of me than a green check mark from somewhere else.

Where the obvious answer runs out

So you write the loop, you fold the path, you land exactly on the root in the file. Feels finished.

It isn't. Look at where each input came from.

The leaf came from your file, and that part is load bearing. It binds the proof to the actual bytes on your disk, so if one byte changes, the leaf changes and the fold stops reaching the root. Good.

But the path and the root both came out of the same document. So the thing you just proved is that the proof file is internally consistent with your file. That's a weaker statement than it feels like, because internal consistency is cheap to manufacture. Given any file at all, you can hash it, build a fresh tree that includes it, and emit a perfectly self-consistent proof today. The fold will check out. It says nothing about time, because nothing in it has been published anywhere.

The root is the only value in that file that was ever committed outside the file. That's the part that carries the timestamp. Everything else is derivation.

So the comparison that matters is not leaf-folded-against-root. It's root-against-chain.

What to look for in your own proof file

Open one. Find the root. Then ask whether the file gives you enough to resolve that root somewhere you don't control: a chain, a transaction identifier, a block. Something you could hand to a block explorer or your own node without asking the issuer's permission.

If the root only ever appears inside the file, and the only way to confirm it was ever anchored is to call the issuer's API and get back a boolean, then the trust boundary hasn't moved. You're back to taking their word for it, with extra hashing in the middle. Correct folding code doesn't save you from that, which is the uncomfortable bit. The crypto can be perfect and the chain of trust can still be a loop.

Where verify-proof sits, and what it won't do

verify-proof is a Python CLI, pip install verify-proof. Three subcommands: hash gives you the SHA-256 of a file, verify checks a proof file against that hash and its Merkle path, and create anchors a hash through ProofLedger, added in 0.3.0. It handles proofs anchored to Polygon and to Bitcoin, including legacy proofs from the retired ProofAnchor service. There's an optional MCP server, pip install verify-proof[mcp], built on FastMCP, exposing five tools, so you can verify or create inside a conversation in Claude Desktop or Cursor.

verify recomputes the hash and walks the path itself rather than asking an API whether the proof is good. Only create touches the network, and it sends the hash, never the file.

Now the limit, because it follows directly from that design decision. Since verification does no network calls, the tool cannot go look at the chain for you. It will do the fold and it will do it locally, on a machine with no route out, with no API key and no account. It will not tell you the root was ever anchored. That step stays yours.

I made that trade on purpose and I think it's the right one, but it is a trade, and I'd rather say so than describe an offline verifier as though it settles the on-chain question too.

The test you can run today

Take a proof file you already have, from whoever issued it.

Cover up the root. Pretend it isn't there.
Hash your file, then fold it up the path with the sibling hashes, using the loop above. Adjust for whatever your format says about side, encoding, and single or double hashing.
Uncover the root and compare.
Then take that computed root and go looking for it on the chain the file names, in an explorer or a node that is not the issuer's.

Read the result:

The fold doesn't reach the stated root. Either your reading of the format is off, or the file is. Fix the format questions first.
The fold reaches the root, but you can't find the root on chain. You have a self-consistent document. You don't have a timestamp.
The fold reaches the root, and the root is sitting in a block. Now you have something, and be precise about what: that digest existed by that block's time. Not that the file is genuine, not that anyone in particular wrote it. Just that those bytes existed by then.

And if step 4 turns into "call our endpoint and we'll confirm it," you've answered the question at the top of this post. The answer just isn't the one you wanted.

verify-proof is free and MIT licensed, published by Fulcrum Enterprises LLC. I built it and I maintain it. Source is at https://github.com/Fulcrum-Enterprises/verify-proof and the package is at pypi.org/project/verify-proof/, if you want something that does the fold in front of you.

Idempotent drafts: keeping a scheduled agent from writing the same post again

Craig Solomon — Thu, 01 Oct 2026 13:00:00 +0000

A scheduler is an at-least-once system wearing an exactly-once costume. You register a job, it fires on a clock, and for a while the mental model holds: the job runs when you said it would, the drafter calls a model, a row lands in the database, you go look at it.

Then reality shows up. The container restarts mid-run. You redeploy during a scheduled window. The host sleeps and the scheduler catches up on missed runs when it wakes. A model call times out at the transport layer, your retry wrapper fires, and the first call had already succeeded on the server side. The job body runs again, and because the job body is "draft something and insert it," you get another row.

Duplicate drafts are not a cosmetic problem. A drafting agent exists to feed a human review step, and review is the scarce resource. Every duplicate row is a decision a person has to make and then unmake. Worse, the duplicates are not identical: same schedule, same prompt, different sampling, so the reviewer cannot even skim and dismiss. They have to read both.

The fix is not a better scheduler. It is making the job body idempotent, so running it again is a no-op instead of a second draft.

The check-then-write trap

The obvious first move is to look before you leap:

def run(conn, platform, slot):
 existing = conn.execute(
 "SELECT id FROM drafts WHERE platform = ? AND slot = ?",
 (platform, slot),
 ).fetchone()
 if existing:
 return
 body = drafter.write(platform=platform, slot=slot)
 conn.execute(
 "INSERT INTO drafts (platform, slot, body, status) VALUES (?, ?, ?, 'ready')",
 (platform, slot, body),
 )
 conn.commit()

This is a time-of-check-to-time-of-use race with a very wide window in the middle, because drafter.write is a network call to a model. Anything that starts a second execution while the first one is waiting on that call passes the SELECT, because the insert has not happened yet. The check is honest and still wrong.

The general fix for this shape of bug is to stop asking the database a question and start giving it a constraint it can enforce.

Derive a key, let the database refuse

Give every draft a key that is a pure function of the work it represents, not of when the work happened:

def draft_key(platform: str, slot: str, recipe: str) -> str:
 return "|".join([platform, slot, recipe])

Three parts, and what goes in each one matters:

platform is the destination. The same idea drafted for different destinations is different work.
slot is the scheduled occurrence, derived from the run's scheduled time, not from the wall clock at execution. scheduled_for.strftime("%Y-%m-%dT%H") gives you a stable string that a catch-up run and the original run both produce. If you use now() here, a catch-up run gets its own key and you are back where you started.
recipe is a version marker for the drafter and prompt. This is the escape hatch: when you intentionally want a fresh draft for a slot you already drafted, you bump the recipe, and the new key is legitimately new.

Keep the key readable if you can. A key you can eyeball in a SQL shell is worth a lot during debugging. If the parts get long, hash them, but then log the plain parts next to the hash.

Then put the constraint where it cannot be raced:

CREATE TABLE IF NOT EXISTS drafts (
 id INTEGER PRIMARY KEY,
 key TEXT NOT NULL,
 platform TEXT NOT NULL,
 slot TEXT NOT NULL,
 status TEXT NOT NULL,
 body TEXT
);

CREATE UNIQUE INDEX IF NOT EXISTS drafts_key ON drafts (key);

If a dashboard reads the same database through Prisma, express the constraint there too so migrations carry it rather than living in a stray .sql file:

model Draft {
 id String @id @default(cuid())
 key String @unique
 platform String
 slot String
 status String
 body String?
}

Claim the slot before you spend the model call

A unique index alone turns the duplicate into a failed insert, which is correct but wasteful: you already paid for the model call before the database told you the work was redundant. Invert the order. Claim the key first, cheaply, then fill in the body.

def claim(conn, key, platform, slot):
 cur = conn.execute(
 "INSERT INTO drafts (key, platform, slot, status) "
 "VALUES (?, ?, ?, 'claimed') "
 "ON CONFLICT(key) DO NOTHING",
 (key, platform, slot),
 )
 conn.commit()
 return bool(cur.rowcount)


def run(conn, platform, slot, recipe):
 key = draft_key(platform, slot, recipe)
 if not claim(conn, key, platform, slot):
 log.info("slot already claimed, skipping: %s", key)
 return
 body = drafter.write(platform=platform, slot=slot)
 conn.execute(
 "UPDATE drafts SET body = ?, status = 'ready' WHERE key = ?",
 (body, key),
 )
 conn.commit()

ON CONFLICT DO NOTHING plus rowcount is the whole trick. The insert either wins or it does not, decided inside the database, and the loser never calls the model. There is no window to race because the claim is one statement.

Two SQLite details make this behave under concurrency. Turn on WAL so a reader (a dashboard rendering the review queue) does not block the writer: PRAGMA journal_mode=WAL. And set a busy timeout on every connection, so a writer that arrives during another write waits for the lock instead of raising immediately. Without the timeout, a claim that collides with an unrelated write surfaces as a locked-database error, and you will misread it as a bug in the claim logic.

Keep writes short, too. Claim, release, do the slow network work outside any transaction, then reopen for the update. Holding a write transaction open across a model call is the fastest way to make a single-file database feel broken.

Where this approach stops helping

It catches exact re-execution of the same unit of work. It does not deduplicate meaning. Bump the recipe and you get a new key and a genuinely new draft, even if the model produces text that reads almost the same as what is already sitting in review. Semantic near-duplicates are a different problem and a unique index will never see them.

It leaves a half-finished state you have to handle. If the process dies between the claim and the update, you keep a claimed row with a null body, and the key is now taken, so retries skip it forever. You need a sweeper that finds claimed rows with no body older than a cutoff you choose and either resets them to unclaimed or deletes them. Pick that cutoff to be comfortably longer than your slowest model call, otherwise the sweeper starts stealing slots from runs that are still working.

It does nothing about ordering or about throughput. Idempotency is not a queue. If you need retries with backoff, priorities, or workers on separate machines, you want a real job store, and at that point SQLite is not the right shared surface.

And it does not solve the actual hard part of a content agent, which is the approval step. A clean, deduplicated queue that nobody reviews is still a pile of unpublished text. The value of the key is that it protects the reviewer's attention, which only matters if there is a place to review.

If you want that half already running instead of assembled from scratch, I build and maintain the AI Content Agent Kit: a Python drafting agent and a Next.js approval dashboard in one docker-compose, MIT licensed.

https://fulcrumenterprises.tech/go/content-agent-kit/?c=devto

stdout is the protocol: what changes when an MCP server moves from stdio to HTTP

Craig Solomon — Wed, 30 Sep 2026 13:00:00 +0000

On stdio transport, an MCP client starts your server as a child process and talks to it over stdin and stdout. That is the whole channel. Stdout is not "where your program prints things while the real protocol happens somewhere else." Stdout is the wire.

Which means this tool is broken:

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("demo")

@mcp.tool()
def list_tables() -> list[str]:
 print("listing tables") # this goes onto the JSON-RPC stream
 return ["users", "orders"]

The function returns the right value. The client still fails, because listing tables\n arrived in the middle of a message stream that only expects framed JSON-RPC. Depending on the client you get a parse error, a tool that never appears, or a server that disconnects during startup with nothing useful on screen.

The failure is confusing because the cause and the symptom are far apart. You added a debug line to a tool and the whole server stopped loading in Claude Desktop.

The fix is one line, and it is about handlers, not about print

Send everything human-readable to stderr:

import sys
import logging

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(logging.Formatter("%(levelname)s %(name)s %(message)s"))
logging.basicConfig(level=logging.INFO, handlers=[handler])

log = logging.getLogger("demo")

@mcp.tool()
def list_tables() -> list[str]:
 log.info("list_tables called")
 return ["users", "orders"]

logging.basicConfig() already defaults to stderr, so you may think you get this for free. You do not, reliably. Whoever calls basicConfig first wins, and if an imported module configures the root logger with stream=sys.stdout, your log lines land on the protocol channel. Configure the handler explicitly in your entrypoint, before you import anything chatty.

The other source of stray bytes is import-time output from a dependency: a banner, a deprecation notice someone routed through print, a progress bar. For Python-level writes you can fence the import:

import sys
import contextlib

with contextlib.redirect_stdout(sys.stderr):
 import chatty_dependency

redirect_stdout swaps sys.stdout for the duration of the block. It does not touch file descriptor 1, so a C extension that writes to fd 1 directly walks straight past it. If you hit that, you are into os.dup2 territory, and at that point it is usually easier to debug over HTTP and keep the dependency out of the stdio path.

A useful habit while writing tools: treat print as a syntax error in server code. Grep for it before you ship.

The switch itself is boring, which is the point

Both transports run the same tool functions. Picking one is an environment decision, not a code decision:

import os

if __name__ == "__main__":
 transport = os.environ.get("MCP_TRANSPORT", "stdio")
 if transport == "streamable-http":
 mcp.run(transport="streamable-http")
 else:
 mcp.run(transport="stdio")

Name the variable whatever you like. The value of doing it this way is that your local Claude Desktop setup and your deployed container are the same artifact, so a tool you debugged locally is the tool that runs remotely.

What is not boring is everything the switch implies.

What changes when you stop being a subprocess

The trust boundary moves. On stdio, your server is a child process of the client, on the user's machine, running with the user's privileges. Nobody else can reach it, because there is nothing to reach. The only thing that drives it is the model sitting in front of one user. On streamable-http, you have a listening socket. Now the interesting question is who can open a connection to it, and the answer is no longer "only the person who launched it."

That is the moment the input guards stop being a nicety. A file tool that resolves paths inside a sandbox root, a SQL tool that rejects anything but reads, an HTTP tool that checks the destination host against an allowlist: on stdio those protect a user from their own model. Over HTTP they protect you from anyone who can reach the endpoint.

Process state becomes shared state. One stdio process serves one client. One HTTP process serves many sessions at once. Module-level globals that felt harmless locally stop being harmless:

# fine as a subprocess, a bug as a server
CURRENT_DIR = None

@mcp.tool()
def set_dir(path: str) -> str:
 global CURRENT_DIR
 CURRENT_DIR = path
 return "ok"

Two callers, one variable. Write tools stateless: take the path, the query, the URL as an argument on every call, and derive everything else from it. If you need a cached resource like a database connection, make it per-request or make it thread-safe on purpose rather than by accident.

Blocking calls start costing other people. Under stdio, a synchronous call that blocks for a while only makes one user wait. Under HTTP, it can stall concurrent requests. Use an async HTTP client for outbound calls, and push blocking work such as a synchronous database driver into a thread with asyncio.to_thread instead of calling it directly inside an async handler.

Debug without the transport at all

The best answer to "I cannot print, so how do I see what is happening" is to stop debugging through the client. Your tool functions are ordinary Python. Call them directly:

import pytest

def test_read_text_file_rejects_escape():
 with pytest.raises(ValueError):
 read_text_file("../../etc/passwd")

No transport, no client, no handshake, and the denial cases are the ones you actually want pinned down. When something misbehaves through Claude Desktop, checking whether the same call misbehaves in a test tells you immediately whether you have a tool bug or a transport bug. Those have completely different fixes and it is easy to spend an afternoon on the wrong one.

Limits

Routing logs to stderr is hygiene, not a feature. It stops a specific, repeatable crash. It does nothing for a tool that returns the wrong data.

An env var that switches transports does not give you authentication, TLS, or rate limiting. streamable-http makes the server reachable. Deciding who may reach it is a deployment question your mcp.run call does not answer, and if you expose an endpoint on the open internet with no auth in front of it, per-call input guards are not going to save you.

Those guards are also input validation, not authorization. They answer "is this call shaped like something allowed" and never "is this caller allowed to see this row." If different callers must see different data, that belongs in a layer that knows who the caller is.

And if you only ever run inside Claude Desktop on your own laptop, the HTTP path may be work you do not need. The reason to wire it early is that retrofitting statelessness into tools written against a single-process assumption is more annoying than writing them stateless on day one.

The MCP Starter Kit is the Python MCP server I built and maintain, with both transports behind one env var, the SSRF, SQL and path guards, and a pytest suite already in place: https://fulcrumenterprises.tech/go/mcp-starter-kit/?c=devto

What has to be online for your artifact verification to pass?

Craig Solomon — Tue, 29 Sep 2026 13:00:02 +0000

I built ProofLedger, which anchors the SHA-256 hash of a file to Polygon and Bitcoin. Almost everything I got wrong while building it was on the verify side, not the anchor side, so that's what this post is about.

Here's the question. When your provenance check goes green in CI, what did it have to reach to get there?

Not "what does it verify." What does it contact. Write down the hostnames. That list is the real trust boundary of your setup, and it's usually shorter than people expect and points at more parties than they expect.

Split the claim into its parts first

A timestamp or attestation check is a few separate assertions stacked into one exit code. Worth pulling them apart, because they have completely different failure modes.

These bytes hash to H.
H was committed to something append-only at some point.
That commitment is real, and I know it's real without the party who issued it telling me so.

Part one is local and cheap. It's just this:

sha256sum dist/app.tar.gz

No network, no auth, no vendor. If the digest doesn't match what the proof says, you're done, the rest doesn't matter. Any verifier that can't do part one without a round trip is doing something strange.

Part two is a data structure question. Either the proof you're holding contains enough material to recompute the commitment, or it doesn't and you have to go ask.

Part three is where the first two stop carrying you. A matching digest and a recomputable commitment both hold even if the commitment exists nowhere but the issuer's database.

"It passes in CI" is not an answer to part three

Green in CI tells you the verifier's dependencies were reachable and returned something the verifier liked. That's it. It doesn't tell you the proof is self-contained, and it doesn't tell you what green would have meant if one of those dependencies had been down.

Two ways that bites.

The first is fail-open. If a verifier treats an unreachable lookup as a soft warning instead of a hard failure, then a green check and a dead endpoint look identical from the outside. You will not notice, because the signal you're watching for is the absence of red. Go read the code path where the HTTP call errors. Does it raise, or does it log and continue? I would check this before I checked anything else.

The second is subtler. Suppose the lookup does work, and the only way to answer part three is an HTTPS call to the issuer's own API that comes back {"valid": true}. That's not verification. That's the issuer's word, restated as JSON, with TLS making you feel better about the transport. If the issuer is also the party whose claim is being checked, you've built a loop. The proof is worth exactly as much as your trust in the company serving that endpoint, which may be plenty, but you should know that's what you bought.

This is why the transparency-log and timestamp-authority conversations in the Sigstore world keep circling back to what a verifier is allowed to require at verify time, and what it does when that thing isn't there. Same question, different vocabulary.

What self-contained actually means

A proof is self-contained when you can get from the file on your disk to the anchored commitment using only material in the proof file, plus one lookup against something public.

If the hash was anchored on its own, that's easy. Hash matches, commitment is the hash, one chain lookup and you're done.

If hashes were batched, you need an inclusion path, and this is the part worth understanding even if you never touch my product, because it's the same mechanic everywhere hashes get batched. The proof hands you the sibling hashes along the path from your leaf to the root, in order. You concatenate and hash, step by step, and you either land on the recorded root or you don't. Nothing in that computation needs a server. The sibling hashes are just bytes. The ordering matters and is part of the proof, and if the format doesn't pin the ordering, the proof is ambiguous and you should say so out loud.

What you cannot derive locally is the root that's actually on chain. That's the one lookup, and it should point at a public chain rather than at the issuer. That's the entire reason I shipped a Python package, verify-proof, that does the check offline and locally, plus a GitHub Action for CI. The public verify URL exists too, and the REST API has a public endpoint with no auth so an auditor or opposing counsel can hit it themselves:

curl "https://proofledger.io/api/v1/verify?hash=$(sha256sum dist/app.tar.gz | cut -d' ' -f1)"

That endpoint is rate limited to 120 requests per hour per IP, which is fine for a human checking a claim and not the thing to build a CI fleet on. The local package is the path for that.

Where this bottoms out, honestly

Offline verification does not mean you conjure chain state out of nothing. It means you can re-check without the issuer. You still have to have obtained the anchored root at some point, from a node you run, a block explorer, or a copy you pinned when you first received the proof. If you never do that, you're trusting whoever handed you the root. Local verification moves the trust from "the vendor's API right now" to "the chain record I can check from any source," which is a real improvement and not a magic one.

Other limits worth stating plainly. A timestamp proves the bytes existed by a certain point. It says nothing about who made them, whether they're correct, or whether the thing they describe is true. Anchoring establishes an upper bound on age and nothing else.

And I'm not claiming my Bitcoin proof is better than a free one. OpenTimestamps gives you free Bitcoin timestamping and the resulting proof is cryptographically just as valid as mine. What I sell is the platform around the proof. The proof itself is the same math available to anyone. On my side, Polygon anchoring is free and unlimited on every plan including the free one, and Bitcoin anchoring is billed per anchor because each one costs me a transaction. I'd rather say that than dress it up.

The test, run it today

Take an artifact you already have provenance for. Not a toy file, a real one from a real build.

Test one. Run your existing verify step with no network at all and record the exit code.

docker run --rm --network none -v "$PWD:/work" -w /work my-ci-image \
 ./verify.sh dist/app.tar.gz
echo $?

If it passes, ask what it actually checked. If it fails, read the error and find out which of the three parts above it failed on. Either answer teaches you something. The bad outcome is a warning line and a zero exit code.

Test two. Append a byte and verify the tampered copy.

cp dist/app.tar.gz /tmp/tampered.tar.gz
printf '\x00' >> /tmp/tampered.tar.gz
./verify.sh /tmp/tampered.tar.gz; echo $?

A verifier that rejects this offline is doing part one correctly. A verifier that needs the network to notice a flipped byte is worth a closer look.

Test three. Feed your verifier a hash that was never anchored. You want an unambiguous "no record," not an empty success.

Test four. Hand the proof to someone outside your org and ask them to check it without talking to you or to your vendor's support. If they can't, the proof isn't portable, and portability is the only property that matters when the check happens somewhere you aren't.

That last one is the question I'd actually like practitioners to answer: when you hand a provenance artifact to an outside party, what do they have to install or trust before they can check it themselves? I have my answer for my own thing. I'm interested in what other formats require in practice.

If you want to poke at mine, the API docs are at proofledger.io/api.html and the OpenAPI spec is at /openapi.json. verify-proof on PyPI is the part you can run without me.

Hard-failing invented numbers in AI-drafted copy

Craig Solomon — Mon, 28 Sep 2026 13:00:00 +0000

A model asked to write promotional copy will eventually produce a figure nobody gave it. Not always, and not obviously. It rounds something it saw upstream, or it fills a slot because the sentence shape wanted a number there, and the result sits in the paragraph looking exactly as credible as the true figures around it. Fabricated numbers are the one class of error that gets harder to spot as the prose gets better.

"Do not invent numbers" in the system prompt is a preference. A function that runs after generation and refuses to return the draft is a guard. This post is about the guard: how to key numeric tokens so the comparison actually means something, where the normalization breaks, and what the check still cannot see.

The check is a set difference, and the hard part is the key

The shape is simple. You have a fact sheet for the thing being written about. You extract every number from the fact sheet, extract every number from the draft, and fail on anything in the second set that is not in the first.

That only works if both sides are reduced to the same canonical form. 1,000 and 1000 are the same claim. 9 and nine are the same claim. $90 and 90% are emphatically not, even though the digits match. So the key has to carry the unit, and the same normalizer has to run over both the fact sheet and the draft.

import re
from decimal import Decimal, InvalidOperation

WORD_NUMBERS = {w: str(i) for i, w in enumerate(
 "zero one two three four five six seven eight nine ten eleven twelve".split()
)}

TOKEN = re.compile(
 r"(?P<currency>[$€£])?"
 r"(?P<value>\d[\d,]*(?:\.\d+)?)"
 r"\s?(?P<unit>%|percent\b)?",
 re.IGNORECASE,
)


def canon(currency, value, unit):
 try:
 number = Decimal(value.replace(",", "")).normalize()
 except InvalidOperation:
 return None
 if unit:
 return f"{number}%"
 return f"{currency or ''}{number}"


def numbers_in(text):
 words = (WORD_NUMBERS.get(w.lower().strip(".,;:"), w) for w in text.split())
 found = {}
 for match in TOKEN.finditer(" ".join(words)):
 key = canon(match.group("currency"), match.group("value"), match.group("unit"))
 if key is not None:
 found.setdefault(key, match.group().strip())
 return found


def unsupported_numbers(draft, fact_sheet):
 allowed = set(numbers_in(fact_sheet))
 return sorted(raw for key, raw in numbers_in(draft).items() if key not in allowed)

The word-number pass runs before the regex, so spelled-out figures get the same treatment as digits. Running it as a token-level substitution rather than a regex on the raw string keeps it from mangling words that contain a number word.

Here it is against a fact sheet and a draft that mostly behaves:

FACTS = """
9 channel definitions, each with its own voice rules and link discipline.
A (product, channel) pair may auto-publish only after 10+ reviewed drafts at 90%+ approval, only on free-API channels, only with the master switch on and credentials set.
MIT licensed.
"""

DRAFT = (
 "It ships nine channel definitions, holds a 90% approval bar "
 "before anything posts itself, and returns $90 of value a month."
)

print(unsupported_numbers(DRAFT, FACTS))
# ['$90']

nine passes because it normalizes to the 9 in the fact sheet. 90% passes on an exact key match. $90 fails, and it fails for the right reason: 90 is in the fact sheet, but as a percentage, and the currency-keyed form has no entry. This is the case a naive digit-scan waves through, and it is the most dangerous one, because a number lifted from a true fact into a false unit is the fabrication a human reviewer is least likely to catch.

Fail, do not fix

Once the check works you have a policy question: when a draft trips it, do you repair the draft or throw it away?

Repairing numbers is a trap. Deleting the offending token leaves a sentence making the same claim with a hole in it, and asking the model to fix the figure invites a second invented one. So the number check hard-fails. The draft does not get published, does not get partially edited, and does not get argued with in code.

A small number of violations are safe to fix mechanically, because the fix carries no semantics. Typographic normalization is the clear case: you can strip characters you never want in output and move on, with no retry and no model in the loop, because removing a dash changes no claim. Keep that list short and keep it strictly separate from the fail list. The rule I would hold to is that anything which could change the meaning of a sentence fails; anything that provably cannot gets sanitized.

Ordering matters too. Sanitize first, then validate, so the validator sees exactly the bytes that would ship.

Make the fact sheet the only source of both

The reason this holds up over time is not the regex. It is that the allowlist is derived from the same structured fact sheet used to build the prompt. One file per product, read by the generator and read by the validator. If you maintain the prompt facts in one place and the validator's permitted figures in another, they drift, and the drift shows up as false failures until someone loosens the check to make the noise stop.

Two more things worth wiring in while you are here, both plain engineering rather than anything clever. Log the reason for every rejection in a form you can read back, and feed recent rejection reasons for that product and channel into the next generation attempt, so the same mistake costs you less each time. And keep the validator flags visible on whatever surface you use to review drafts, so a human approving something can see what the check looked at.

What this does not catch

The honest limits, because they decide whether the check is worth building for your case.

It only sees numbers. A draft claiming an integration that does not exist, a customer you do not have, or a capability you never shipped passes cleanly. Those need their own guards, and some of them need a person.

It sees numbers, not relations. Every figure can be present in the fact sheet and the sentence can still be false: a true count attached to the wrong noun, a true percentage attributed to the wrong condition, a real figure placed in a comparison you never measured. The check confirms provenance of tokens, not truth of sentences.

Magnitude suffixes and ranges need explicit handling. 2k, 2 thousand, and 2,000 are one claim in three spellings, and a range like 10 to 20 is two tokens that may both be absent from the fact sheet even when the range is fine. The version above handles none of that. Extend the canonicalizer, do not extend the allowlist.

Word-number expansion buys false positives. "One of the channels" becomes a bare 1 in the scan. You can narrow it by only expanding word numbers directly preceding a noun, or you can accept the noise and let the fact sheet carry the small integers. Either choice has a cost, and the choice with no cost does not exist.

Carve-outs are where fabrications hide. Skipping code blocks, URLs, and version strings makes the check usable, and every skipped region is a place an invented figure can sit. If you exclude something, exclude it narrowly.

And if your copy carries no figures at all, this guard buys you very little. It earns its keep specifically when a generator writes about things with counts, prices, and percentages attached, and when the cost of one wrong figure under your name is higher than the cost of maintaining a fact sheet.

Content Agent Pro is the drafting engine I built and run on my own store, with deterministic validators including this one, the reject-reason feedback loop, and per-channel graduation gates already wired together: https://fulcrumenterprises.tech/go/content-agent-kit-pro/?c=devto

The allowlist in your MCP call_api tool should match hosts, not strings

Craig Solomon — Sun, 27 Sep 2026 13:00:00 +0000

The moment you expose a tool like this over MCP, the URL stops being yours:

@mcp.tool()
async def call_api(url: str, method: str = "GET") -> str:
 """Call a REST API and return the response body."""
 async with httpx.AsyncClient() as client:
 r = await client.request(method, url)
 return r.text

The model fills in url. Your server makes the request, from inside your network, with whatever egress your process has. That is a request forwarder with a language model at the wheel, which is the classic shape of a server-side request forgery bug. The difference from a normal SSRF is that the attacker input does not have to arrive in an HTTP parameter. It can arrive in a document the model read, a webpage it fetched a second ago, or a support ticket it was asked to summarize.

So you add an allowlist. Here is the one I wrote for my own server, and what I had to stop it from doing.

The string check that does not hold

The first instinct is to check the text of the URL:

if not url.startswith("https://api.example.com"):
 raise ValueError("host not allowed")

Three URLs pass that check and none of them go where you think.

https://api.example.com.attacker.tld/collect
https://api.example.com@attacker.tld/collect
https://api.example.com.attacker.tld/../../whatever

The first is a suffix trick: api.example.com.attacker.tld is a hostname under attacker.tld, and it starts with your allowed string. The second is the userinfo field. Everything before the @ in the authority is a username, so the host is attacker.tld and api.example.com is just a decorative login name. A "api.example.com" in url check is worse again, because the substring can live in the query string of a completely different URL.

The fix is to stop reasoning about the URL as text and let a parser tell you what the host actually is.

Parse, then compare

from urllib.parse import urlsplit

ALLOWED_HOSTS = {"api.example.com", "api.internal-billing.example.com"}
ALLOWED_SCHEMES = {"https"}


def check_url(url: str) -> str:
 parts = urlsplit(url)
 if parts.scheme not in ALLOWED_SCHEMES:
 raise ValueError(f"scheme not allowed: {parts.scheme!r}")
 host = parts.hostname
 if host is None or host not in ALLOWED_HOSTS:
 raise ValueError(f"host not allowed: {host!r}")
 return url

urlsplit(...).hostname is doing real work here. It strips the userinfo, strips the port, and lowercases the result, so HTTPS://API.EXAMPLE.COM@evil.tld/ gives you evil.tld and the check fails where it should.

The scheme check matters as much as the host check. Without it, file:///etc/shadow has no host at all, and depending on your client stack a scheme you never considered can still open something. Allow the schemes you actually use and reject the rest by default.

Set membership is an exact match, which is the behaviour you want. If you need subdomains, write the rule out rather than reaching for a substring:

def host_allowed(host: str, allowed: set[str]) -> bool:
 return any(host == a or host.endswith("." + a) for a in allowed)

Be honest about what that widens to. Anyone who can create a subdomain on that parent domain is now inside your allowlist, and on a large SaaS provider that can be anyone with an account.

Layers you can add on top

The host allowlist is the boundary. A few standard-library techniques harden the path around it, and you can bolt each of these on independently.

Reject non-public addresses. If your allowlist is generated from config and someone adds a wildcard, or you allow a host you do not control, resolving before you connect catches the address ranges that hurt: loopback, private ranges, and the link-local range where cloud instance metadata lives.

import ipaddress
import socket


def check_resolves_public(host: str) -> None:
 for *_, sockaddr in socket.getaddrinfo(host, None, proto=socket.IPPROTO_TCP):
 ip = ipaddress.ip_address(sockaddr[0])
 if (ip.is_private or ip.is_loopback or ip.is_link_local
 or ip.is_reserved or ip.is_multicast):
 raise ValueError(f"{host} resolves to a non-public address: {ip}")

ipaddress handles the encodings by hand-rolled checks miss. IPv4 written in decimal, IPv6, and IPv4-mapped IPv6 all parse into an object whose is_private and is_loopback properties tell you the truth.

Do not follow redirects blindly. An allowed host that returns a redirect can hand the request to a host you never allowed, and most HTTP clients will follow it for you by default. Turn that off and re-run the check on every hop:

async def fetch(client, url: str, max_hops: int):
 hops = 0
 while True:
 check_url(url)
 r = await client.request("GET", url)
 if not r.is_redirect:
 return r
 hops += 1
 if hops > max_hops:
 raise ValueError("redirect limit exceeded")
 url = str(r.next_request.url)

The client is constructed with follow_redirects=False so the loop above is the only thing moving between hops.

Keep credentials out of the model's reach. If the tool signature takes a headers dict, the model can set Authorization on a request to any allowed host, and it can also fail to set it. Injecting auth server side, keyed on the resolved host, means the token never appears in a tool argument and never appears in a transcript.

What this does not solve

A host allowlist controls where the request goes. It controls nothing else, and the gaps are worth naming.

It does not close the gap between the check and the connection. You resolve a name, you decide it is fine, and then the HTTP client resolves it again when it opens the socket. A DNS record with a short TTL can answer differently on the second lookup, which is DNS rebinding. Closing that means pinning the address you validated and connecting to that address directly, which is a transport-level change, not a text-level one.

It does not stop exfiltration to an allowed host. If your allowlist includes an API that echoes a path or accepts arbitrary POST bodies, data can leave through it. Path and method restrictions are a separate control.

It does not make the response safe. Whatever comes back is going into the model's context, and an allowed host serving attacker-influenced content is a prompt injection channel. Treat tool output as untrusted input to the next turn.

And it is the wrong tool if your product genuinely needs to fetch user-supplied URLs across the open internet. An allowlist cannot enumerate the internet. That case belongs to an egress proxy or a network policy, with the fetcher running somewhere that cannot reach anything internal in the first place.

Test the denials

The check above has one job and it is a refusal. Write the test file as the list of things that must be rejected: the suffix host, the userinfo host, the non-HTTPS scheme, the file:// URL, the redirect to an off-list host. A passing request proves very little. A rejected one proves the boundary exists.

The MCP server I built and run packages this host allowlist together with a read-only SQL guard and a path-traversal sandbox as a Python template you can clone: https://fulcrumenterprises.tech/go/mcp-starter-kit/?c=devto

What survives if the company that issued your timestamp disappears

Craig Solomon — Sat, 26 Sep 2026 13:00:00 +0000

Here's the question I'd ask about your own release pipeline, and it isn't "is the timestamp signed."

It's this: if the service that issued your timestamps went dark tonight, what could you still prove in the morning?

The answer doesn't depend on what the service promised. It depends entirely on what you kept.

The obvious answer, and why it falls over

Obvious answer: you kept the verify link. Every timestamping service hands you one. You stash it in the release notes, you move on.

But a verify URL is a request to the issuer. That's the whole thing it is. If the issuer is gone, that URL is a 404, and what you're left holding is a screenshot of a page that used to load. A screenshot is not evidence of anything except that you once had a browser.

So the link is fine as a convenience. It is not the proof. Worth separating those two in your head before you go further.

What a timestamp actually consists of

Strip it down and there are two independent pieces.

The binding. A hash ties a specific sequence of bytes to a short digest. You run SHA-256 over the file, you get a digest, and anyone who has the same bytes gets the same digest. Forever. No service is involved in that step and none can be. This is the part you already control, and keeping it costs you nothing beyond keeping the file.

The anchor. Something append-only and expensive to rewrite says "this digest was known by this point." That's the part you're renting from somebody, and that's the part that goes away when they do.

So the anchor has to end up on your disk as data you hold, not as a pointer into someone else's database.

The part you have to store yourself

Here's what you actually need on your own disk for the anchor to survive the issuer:

The digest. You can always recompute it from the file, but store it anyway, because it's tiny and it tells you which file you meant.
Which chain the anchor went to.
The transaction id on that chain.
The path from your digest to whatever that transaction committed to.

Number four is the one that gets dropped, and it's the one that matters.

Anchoring one hash per transaction is expensive, so batching is normal. Many digests go into a Merkle tree, the root of that tree goes on chain, and the transaction commits to the root, not to your digest. Which means the on-chain record says nothing about your file on its own. Your digest plus the root is not a proof. You need the sibling hashes along the path from your leaf up to the root, in order, with the left/right position of each step.

With those, verification is arithmetic you can do yourself:

leaf = sha256(file_bytes)
node = leaf
for (sibling, side) in path:
 node = sha256(sibling + node) if side == "left" else sha256(node + sibling)
assert node == root_in_transaction

That loop is the entire proof. It needs no network, no account, no vendor. If you're storing a verify URL instead of that path, you don't have a portable proof, you have a bookmark.

Two chains, one digest, and an honest caveat

ProofLedger, which is mine, sends the same digest to Polygon and to Bitcoin. Same hash, two independent records. The reason is redundancy of the anchor, not strength of the cryptography.

I want to be blunt about that second part because it's where this category gets oversold. OpenTimestamps gives you Bitcoin timestamping for free and the proof it produces is cryptographically just as valid as mine. There is no version of this where I have a better Bitcoin proof than a free tool. A hash committed to a chain is a hash committed to a chain. What differs between options is the platform wrapped around it: what you can query, what you can export, what the verification path looks like for someone who doesn't trust you. Anyone selling you a stronger proof is selling you an adjective.

Verification has to be boring, and it has to be public

The test of whether a timestamp is worth anything is whether the person relying on it can check it without asking you. Not without asking the vendor. Without asking you, the party with the motive.

That's why the verify endpoint on my API takes no auth at all:

H=$(shasum -a 256 release-artifact.tar.gz | cut -d' ' -f1)
curl "https://proofledger.io/api/v1/verify?hash=$H"

No key, no account, 120 requests per hour per IP. An auditor or opposing counsel can run that from a laptop. The whole v1 API is three endpoints: POST /api/v1/proof to submit a hash with a Bearer sk_ key, GET /api/v1/proof/:id for status plus chain_tx and bitcoin_txid, owner only, and that public verify. There's an OpenAPI spec at /openapi.json if you'd rather generate a client than read docs.

The file itself never goes anywhere. You hash locally, you send the digest. That's not a privacy feature bolted on, it's just what the engine does, because the engine only ever sees a digest. It has no idea what kind of file you hashed and there's no per-file-type pipeline behind it.

For the offline case there's a verify-proof package on PyPI that does the same check locally, and a GitHub Action so CI can do it on every build.

The limits, stated plainly

Reading a chain means reading a chain. Local verification confirms your bytes match your proof material without involving me, but confirming the anchor itself still means somebody queries a node or an explorer. That's a real dependency and I'd rather name it than let "offline" do work it can't do.

A timestamp proves existence by a point in time. It does not prove authorship, it does not prove the contents are true, and it does not prove you're the one who made the file. It proves the bytes existed. That's a narrow claim and narrow is what makes it hold.

And a tradeoff I took that I'm not thrilled about: the Merkle-inclusion download sits on the Business tier. Polygon anchoring is free and unlimited on every plan including the free one, and every plan has API access with a monthly cap, but that particular export is gated. That's a business decision, not a technical one, and I'd rather say so than have you find out later.

The test to run today

Pick an artifact you shipped and still care about. Then try to verify its timestamp using only files on your own disk and a public block explorer. No vendor login, no support ticket, no dashboard.

Can you get from the bytes to a confirmed on-chain record by yourself? If yes, you have a timestamp. If you have to authenticate to somebody first, you have an account with a company that has a timestamp.

Run that against whatever you're using now, including mine. proofledger.io, API docs at proofledger.io/api.html. I build and run it.

Genuine question for anyone doing this at scale already: what do you actually store next to your artifacts, just the digest and a txid, or the full inclusion path? I keep going back and forth on what belongs in the repo versus what belongs in cold storage, and I'd like to hear how other people landed on it.

Turning thumbs up and down into keyword weight proposals

Craig Solomon — Fri, 25 Sep 2026 13:00:00 +0000

A keyword scorer is the cheapest filter you can put in front of an LLM. You write terms, you give them weights, you sum the matches, and anything over a threshold gets read by the expensive model. It works on day one. The problem starts on day thirty, when your product has moved and your term list has not.

The fix is not to rewrite the list by hand every month. It is to capture the feedback you are already collecting and turn it into weight proposals you can approve or reject. Here is the mechanism, including the part that breaks if you skip it.

Log attribution at scoring time, not later

This is the step that decides whether any of the rest is possible.

When you score an item, you know exactly which terms fired and what each one contributed. If you store only the final score, that knowledge is gone, and later you are reduced to re-running a guess about which term earned the hit.

# weights: term -> {"dimension": "threat", "weight": 3.0}
DIMENSIONS = ("opportunity", "adoption", "pivot", "threat")

def score(text, weights):
 lowered = text.lower()
 totals = {d: 0.0 for d in DIMENSIONS}
 hits = []
 for term, cfg in weights.items():
 if term in lowered:
 totals[cfg["dimension"]] += cfg["weight"]
 hits.append((term, cfg["dimension"], cfg["weight"]))
 return totals, hits

Persist hits next to the item:

CREATE TABLE signal_terms (
 item_id INTEGER NOT NULL,
 term TEXT NOT NULL,
 dimension TEXT NOT NULL,
 weight REAL NOT NULL,
 PRIMARY KEY (item_id, term)
);

Now a thumbs up or down on an item is a labelled example for every term that fired on it. Without this table you have a rating on a piece of text and no way to attach it to anything you can tune.

Split the credit, do not hand it out twice

The naive aggregation gives every term that fired a full vote. That quietly rewards your broadest terms. A generic word that appears on almost everything gets credited for every good item your sharp terms actually found, and its weight creeps up until it dominates the score.

Split the credit across the terms that fired on that item, proportional to what each contributed:

from collections import defaultdict

def term_feedback(rows):
 """rows: (item_id, term, weight, verdict) with verdict in {"up", "down"}"""
 by_item = defaultdict(list)
 for item_id, term, weight, verdict in rows:
 by_item[item_id].append((term, weight, verdict))

 up = defaultdict(float)
 down = defaultdict(float)
 for entries in by_item.values():
 total = sum(w for _, w, _ in entries) or 1.0
 for term, weight, verdict in entries:
 credit = weight / total
 if verdict == "up":
 up[term] += credit
 else:
 down[term] += credit
 return up, down

Votes are now fractional, which is the honest representation. A term that was one of several reasons an item scored is one of several reasons it was good.

Smooth before you rank

A term with one thumbs down and nothing else has a raw precision of zero. Act on that and you will delete a good term because of a single bad morning.

Add a prior, then require a minimum amount of evidence before a term is eligible for a proposal at all:

PRIOR_UP, PRIOR_DOWN = 1.0, 1.0

def precision(u, d):
 return (u + PRIOR_UP) / (u + d + PRIOR_UP + PRIOR_DOWN)

The prior pulls thin evidence toward the middle, so terms with little feedback sit in the do-nothing band on their own. The minimum-evidence gate is still worth having, because it makes the reason a term was skipped explicit in the code instead of implicit in the arithmetic.

Propose a diff, do not apply one

Here is the design decision worth arguing about: the tuner should emit a proposal, not a write.

MIN_EVIDENCE = 5.0
STEP = 0.25
FLOOR, CEIL = 0.5, 5.0
UP_BAND, DOWN_BAND = 0.70, 0.40

def propose(up, down, weights):
 out = []
 for term in set(up) | set(down):
 u, d = up[term], down[term]
 if u + d < MIN_EVIDENCE:
 continue
 p = precision(u, d)
 current = weights[term]["weight"]
 if p >= UP_BAND:
 new = min(current + STEP, CEIL)
 elif p <= DOWN_BAND:
 new = max(current - STEP, FLOOR)
 else:
 continue
 if new != current:
 out.append({"term": term, "from": current, "to": new,
 "precision": round(p, 2), "up": round(u, 1),
 "down": round(d, 1)})
 return sorted(out, key=lambda r: r["precision"])

Three reasons to keep a human in the loop.

Keyword weights are coupled. Move several at once and you have not adjusted terms, you have moved the whole score distribution relative to your threshold. A reviewer looking at the batch notices that. A loop applying its own output does not.

Feedback is an opinion about relevance, and relevance is a business judgement. The person who knows whether "seed round" should matter more this quarter is the person reading the briefing, not the aggregator.

An auto-applying loop has no audit trail. When scoring goes strange, you want a list of accepted proposals to walk backwards through, not a weights file that has been drifting on its own.

The fixed step and the floor and ceiling do the rest of the safety work. No single round can move a term far, and nothing can decay to zero or run away.

The loop can measure precision and not recall

Your feedback is not a random sample. You only see thumbs on items that survived the filter and got delivered. Every item the keyword pass dropped is invisible, so the loop can measure precision and cannot measure recall at all.

Run it long enough and the tuner sharpens the terms you already have while the things you never wrote down stay unreachable. Precision goes up. Coverage quietly narrows.

The cheap mitigation is a holdout. Route a small random sample of below-threshold items into a review queue on a schedule, label them like anything else, and look at what comes back marked good. Those items are the only direct evidence you get about what your term list is missing. Weight tuning cannot fix a missing term; only a human reading a near-miss can add it.

What this does not solve

It does not invent vocabulary. A new competitor's name, a new framework, a term of art that appeared last week: none of that arrives from tuning weights on terms you already wrote.

It drifts toward whatever you happen to click. If you only rate the items that annoy you, you are training an annoyance detector.

It needs a cold start. Until terms clear the evidence gate, the tuner correctly proposes nothing, and a system that appears to do nothing for a while is a system people stop feeding.

And if you will not give feedback, skip the whole thing. A tuner with no labels is a scheduled job that reads an empty table. Static weights you revise by hand are a perfectly reasonable alternative, and honest about what they are.

This auto-tuning loop is one piece of the Market Radar Kit, a self-hosted market-intelligence agent I built and run in Docker against my own Claude subscription: https://fulcrumenterprises.tech/go/market-radar-kit/?c=devto