DEV Community: Patrick Hughes

AI Agent Memory: What Actually Works in 2026

Patrick Hughes — Sun, 28 Jun 2026 14:45:17 +0000

AI agents forget. They also hallucinate that they remembered. The result is the same: you open a file or check a ledger and the thing the agent swore it wrote is not there.

The fix is not a bigger vector database. It is a tiny set of durable surfaces plus deterministic checks that run on every "done" claim.

The surfaces that survive

Keep memory in plain files the operator can read without a special viewer.

Append-only logs (log.md, run-ledger rows, event streams). Newest at top. One line per real event when possible.
Knowledge wiki with layers (sources, entities, concepts, syntheses). Write sources once. Touch entities on every new signal.
Queue and Reports. The task files and the proof they produced.
Published artifacts and frontmatter. The only place that counts is the final output location.

Everything else is cache or scratch. If it is not in one of these, it did not happen for the purpose of later work.

How we actually use it

Every scheduled agent writes to the ledger on start and finish. The think checker reads the ledger's declared artifacts and compares them to disk and DB. No "the agent said it was green."

Vault health and nightshift linters walk the wiki for orphans and contradictions. They repair what they can and surface the rest.

The digest agent writes a source page for each article and links it into 10+ existing pages. That compounds.

When an agent claims it updated a file, the claim is recorded and a later pass asserts the bytes are there.

What fails in practice

Long context windows. The model predicts the next token, not the prior state. After 30 turns it confidently "remembers" a step that was overwritten.

Vector stores without grounding. They return plausible passages. You still have to open the file to know if the patch landed.

Shared mutable state without locks. Two agents write the same report. The second clobbers the first. The operator sees one version and assumes the story is complete.

No TTLs. Old drafts, stale ideas, and resolved requests sit forever and pollute search.

The rules that keep it small

One line per log entry. If it needs more, link to a report.
Sources are append-only. Never edit a past source page.
Every completion that mutates state records a falsifiable claim.
Every claim is re-checked by an independent pass (claims checker, think, doctor).
Drafts in Queue have TTLs. Approved drafts in Review have age gates. Old stuff moves or dies.
The operator can always cat the file and understand the state without running the agent.

When fancy memory is a trap

If you are building multi-agent research loops with long retrieval chains, you may need embeddings and rerankers. For a one-person holdco that ships one artifact per day, the cost in maintenance exceeds the gain.

Files + git history (for code) + the structured event/claim layer + simple date folders beat a custom memory service 95% of the time. You can read them on a plane with no wifi.

What we ship instead

We ship the checker, not the memory bank.

brain think --heal exists because the ledger said the autopublisher ran but the published count was zero. The heal path wrote the post, ran the sub-QA, posted it, and left the proof in Published/ and the live URL.

The memory that matters is the one a script can query at 07:30 and decide whether the thing that should have shipped actually shipped.

Accompanying prompt

What the prompt does: Audits an agent workflow and turns vague memory into plain-file state, append-only events, and checks that prove each completion claim.

You are auditing the memory layer for an AI agent workflow.

Your job is to replace vague "the agent remembers" claims with durable state that a human can inspect and a script can verify.

Work in five passes:

1. List every place the agent currently stores state. Separate durable files, append-only ledgers, databases, caches, prompts, and scratch output.
2. For each surface, answer: who writes it, who reads it, what proves it is current, and what happens if two agents write at once.
3. Find any completion claim that is not backed by a file, ledger row, API readback, test result, or live URL.
4. Propose the smallest durable memory system that would survive a cold restart: plain files first, append-only events second, database rows only where they are already part of the product path.
5. Write the exact checks that should run before an agent is allowed to say "done."

Return:

- A table of current memory surfaces and their risk.
- A short list of state that should be deleted, archived, or ignored.
- The proposed durable memory contract.
- The verification commands or scripts that prove the contract.
- One example "done" claim rewritten as a falsifiable claim.

Constraints:

- Prefer files a senior engineer can open and read.
- Do not add a vector database unless the workflow has a specific retrieval failure that plain files cannot solve.
- Do not trust model output as evidence.
- Every important claim needs a deterministic readback.

Copy the block above.

See the full set of checks in config/think/check.py and config/claims/check.py. When an agent tells you it is done, make it prove it with a file you can open.

https://bmdpat.com/tools/agentguard

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

What Anthropic's MITRE ATT&CK report means for solo AI builders

Patrick Hughes — Thu, 25 Jun 2026 21:31:09 +0000

What Anthropic's MITRE ATT&CK report means for solo AI builders

Anthropic just published a year of cyber threat intelligence. They mapped 832 banned accounts to the MITRE ATT&CK framework. Co-released with the Verizon 2026 DBIR, it is the most authoritative look at how people actually misuse frontier models for hacking.

For a solo builder shipping AI features or agents, this report is a glimpse into the future of your own threat model. You do not need to read all 40 pages. You just need to know how the shift from "text generation" to "agentic action" changes what you have to protect.

1. Malware writing is the floor, lateral movement is the ceiling

The headline number is that 67.3% of misused accounts used Claude for malware writing. This is not surprising. AI is very good at writing code, and that includes malicious code. If an attacker wants a python script that scrapes a specific site or a bash command that finds open ports, the model will help them.

But the statistic that should keep you awake is the 6.5% of accounts used for lateral movement.

In a traditional attack, lateral movement is a manual, labor-intensive process. Once an attacker gets a foothold in a network, they have to spend hours or days exploring, discovering accounts, and trying to escalate their privileges. It is a slow, human-driven game.

AI is changing that. Attackers are now using models to automate the discovery and navigation of compromised systems. They are moving deeper into the kill chain with real-time decisions made by the model.

The takeaway for you: your agent's blast radius matters more than your input filter. You can spend weeks hardening your system prompt to prevent "jailbreaks," but if an attacker finds one crack, the model itself can now help them navigate your entire stack. If your agent has access to your Supabase keys or your Vercel environment variables, the model can help the attacker find and exploit those connections faster than any human could.

2. MITRE ATT&CK does not capture agentic orchestration

MITRE ATT&CK is the standard framework that underwrites almost every enterprise security center in the world. It provides a common language for describing how attacks happen. But Anthropic explicitly noted that the current framework is being outgrown. It does not yet capture agentic orchestration.

When an attacker chains multiple stages of an attack together with minimal human input, they are operating past the edge of traditional security models. They are not just using a tool; they are running an autonomous campaign.

If you are shipping agents that can plan and execute multi-step tasks, you are operating in this same territory. You cannot rely on standard security frameworks to describe your risk. You have to think about the autonomous runway you give your models. If an agent can run for hours without a human in the loop, that is a window of opportunity for an attack to go from a minor incident to a total database wipe.

The report shows that medium-to-high-risk actors grew from 33% to 56% of the banned accounts in just six months. The mix of attackers is getting more dangerous. They are concentrating their AI use on the operationally hard parts of an attack, not the easy parts.

3. Inference-time safeguards raise the defensive floor

There is some good news. Anthropic is deploying cyber safeguards directly inside the model layer. They call this Project Glasswing. It means the model itself is trained to detect malware development, exfiltration patterns, and reconnaissance at the moment of inference.

For a solo builder, this is a massive advantage. Staying on a managed frontier API like Claude gives you a defensive uplift that you cannot get with a local model. You get a team of security researchers working to stop your users (or your compromised agents) from doing the worst things before the call even finishes.

Uncensored local models have their place, but for production agent work where the model has access to your infrastructure, a managed API with built-in safeguards is the rational choice. It raises the floor of what an attacker can force your agent to do.

The cheapest control: Runtime guardrails

You do not need a nation-state security budget or a 50-person security team to protect your work. You need concrete, simple controls at the call site.

The Anthropic report shows that the threat is moving toward "autonomous orchestration." The best defense against an autonomous threat is an autonomous limit.

Runtime guardrails are the cheapest way to bound an agent's runway. Budget caps, token limits, and rate enforcement are not about stopping a sophisticated hacker. They are about limiting how much damage a compromised agent can do before you notice.

If an agent is tricking your system into a loop or trying to exfiltrate your entire database, a budget guard or a loop guard will kill the run in seconds. It turns a potential disaster into a small, recorded incident in your logs.

If you want to start guarding your agent loops today, check out AgentGuard. It is a local-first SDK that adds these hard stops to your Python agents with a few lines of code.

Protect your runway. Don't let an autonomous agent turn a small bug into a big bill.

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

AI Agent Memory in 2026: How It Works and When to Use It

Patrick Hughes — Thu, 25 Jun 2026 14:45:10 +0000

AI Agent Memory in 2026: How It Works and When to Use It

Most agent demos forget everything between calls. That works for toy scripts. It breaks the moment you want an agent that improves over a week of work.

Memory is not one thing. It is several different stores that solve different failure modes.

Short-term context vs long-term recall

The context window is your agent's working memory. It is fast and expensive. Keep it for the current task only.

For anything that spans sessions you need retrieval.

Vector stores are the current default. Embed past steps, tool results, and user feedback. Retrieve the top-k relevant chunks when the agent starts a new step.

They are good for semantic similarity. They are bad at exact sequences and time.

Episodic memory

Store the actual trace: "on June 20 at step 4 I called the pricing API and got 429, then retried with backoff".

This is gold for debugging and for the agent to avoid repeating the same mistake.

A simple JSONL file or a small SQLite table works on consumer hardware. No fancy embedding required for the first version.

Persistent state

Some agents need durable facts.

"The user's preferred region is eu-west-1"
"Last successful backup was at 2026-06-23T14:12Z"

Put this in a key-value store or a small Postgres. Update it explicitly when the agent learns something trustworthy.

Do not trust the LLM to remember it correctly inside the context.

When to add each layer

Start with good system prompts and short context.

Add vector retrieval when the agent needs to reference past research or documentation.

Add episodic traces when you see it repeating the same errors across runs.

Add persistent facts when user preferences or long-running state actually matter.

The goal is not maximum memory. The goal is the smallest memory surface that makes the agent reliable for the job.

Most production agents I have shipped use two or three of these stores. Never all of them at once until the pain was real.

If you are building agents that run for days or weeks, memory design is the difference between a demo and something you can trust overnight.

Ready to build your own reliable AI agents with proper memory? Start with AgentGuard: https://bmdpat.com/tools/agentguard

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Your AI Agent Says "Done." Make It Prove It.

Patrick Hughes — Wed, 24 Jun 2026 14:45:08 +0000

Last week one of my agents reported that it had logged a new vendor in my Vendor Map file. Clean run. No error. The summary said done. I opened the file. The vendor was not there. Nothing crashed. The agent simply believed it finished work it never did.

AI agents routinely report tasks as complete that were never actually done. A clean exit code and a "done" message prove the agent finished running, not that the work landed. The fix is to treat every reported completion as a falsifiable claim, then run a deterministic checker that verifies each claim against the real file, frontmatter, or command output before you trust it.

Why do AI agents report work they didn't do?

Because the model is predicting a plausible ending, not checking reality. When an agent writes "done," it is generating the most likely next sentence given a transcript that looks like finished work. That is not the same as opening the file and confirming the change is there.

This gets worse the longer the session runs. The agent edits, gets interrupted, edits again, loses track of which write actually committed, and signs off confident. I have watched Claude, Codex, and my own scheduled agents all do it. The failure is not laziness. It is that "I did X" is cheap to say, and the model has no built-in cost for saying it falsely.

What is the difference between a log line and proof?

A log line is a statement. Proof is a check against the world.

"Logged the vendor in Vendor Map" is a statement. It costs nothing and it can be wrong. Proof is: open Vendor Map, search for the vendor name, confirm the line exists. One of those you can automate. The other you have to trust. The whole problem is that we keep trusting the statement and calling it proof.

What does a falsifiable claim look like?

A falsifiable claim is a completion report written so a script can prove it false. Not prose. A small, fixed grammar.

When one of my agents reports it changed something, it has to record the claim in a structured form:

file_contains: this file now contains this string
path_moved: this file moved from here to there
frontmatter: this file's frontmatter field equals this value
glob_count: this many files match this pattern
command: this command, run under a locked directory, exits clean

If the agent cannot phrase its "done" as one of those, it does not get to call the work done. The constraint forces honesty. Vague claims are exactly the ones that hide false completions.

How do you check a claim automatically?

You write a checker that reads the claims and tests each one. Mine appends every claim to a daily file, and a small script walks the list. file_contains opens the file and greps. path_moved confirms the old path is gone and the new one exists. frontmatter parses the field and compares. command runs and checks the exit code.

If every claim holds, the report is green. If one fails, the report is red and names the claim that lied. The session does not get to mark itself complete until the checker passes. That single rule, completion is legitimate only after the checker is green, is what catches the vendor-map class of bug.

What happens when a claim turns out false?

The report goes red and tells me which claim failed. That is the point. A false "done" used to slip through silently and surface days later when I went looking for work that was never there. Now it surfaces the same day, with the exact claim that did not match reality.

I keep the claims narrow on purpose. A free-form opinion ("this approach is better") carries no check and is recorded but not verified. Only falsifiable statements get tested. You cannot verify a judgment, but you can always verify "this file contains this string."

Where does this save you the most?

Anywhere an agent runs unattended. A coding agent that says it added a test. A nightly job that says it updated a report. A fleet of scheduled tasks reporting completion while you sleep. The longer the gap between the claim and the moment you check it, the more a false "done" costs you.

The principle is the same one behind every external guard you put on an agent: do not trust the agent's self-report, enforce reality from outside. Claims-checking enforces that the work happened. The other half is enforcing that the work did not cost more than it should.

That is the gap AgentGuard fills on the spend side. It wraps an agent with a hard budget, token, and rate ceiling, so a confident agent that decides to retry all night stops at a limit instead of draining your account. Same idea as claims-checking, pointed at cost: don't trust, enforce. Install it free with pip install agentguard47, or read how it works at https://bmdpat.com/tools/agentguard

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Give Your AI Agents an Append-Only Event Log

Patrick Hughes — Mon, 22 Jun 2026 14:45:08 +0000

Your AI agent finished its run. The exit code was zero. The log says done. But did it actually do the work? Most agent setups cannot answer that. They store the last state and nothing about how they got there.

An append-only event log records every start, finish, and lock as a separate timestamped line that is never edited or deleted. Because the log is immutable, you can replay it to reconstruct exactly what your agent did at any point in time. This catches crashed runs and stuck locks that a status field alone hides.

Why can't you tell what your agent actually did?

Status fields lie by omission. A row that says status: done tells you the final answer, not the path. If the agent crashed halfway and a later run overwrote the row, the crash is gone. You see green. You shipped nothing.

I hit this building the brain that runs my one-person company. A scheduled task would die mid-run, the next run would stamp a fresh status, and the morning report showed everything fine. The state was current. The history was a lie.

What is an append-only event log?

It is a file you only ever add lines to. Never edit. Never delete. Each line is one event: what happened, when, and which run or lock it belongs to. I write one JSONL file per day at Reports/Events/2026-06-22.jsonl. One event per line. New events go at the bottom.

The rule is the whole point. Mutable state answers "what is true now." An immutable log answers "what happened, in order." You need both, and most setups only keep the first.

Isn't this just logging?

Logs are for humans to read after something breaks. An event log is structured data a program reads to make a decision. The difference is the schema. Each event has fixed fields (type, timestamp, run id) so a checker can fold them without parsing prose.

Plain logs also get rotated, truncated, and edited. The append-only rule is a promise: this record is complete and ordered. You can build automated checks on that promise. You cannot build them on a log file that some cleanup job trims every week.

What events should an agent emit?

Start with two pairs. Every run emits run.start when it begins and run.finish when it ends. Every lock emits lock.acquire when taken and lock.release when freed. That is four event types and it already covers the failures that hurt most.

I wired this into one shared include that every scheduled task already loads, so I added zero lines to any individual agent. The instrumentation lives in one place. Every runner got it for free.

How do you reconstruct state from events?

You read the log top to bottom and fold each event into a running picture. A run.start with no matching run.finish means a run that never ended. A lock.acquire with no lock.release and a dead process means a stuck lock.

I built a replay --at <timestamp> command that stops folding at any instant and prints the system as it stood right then. When something broke at 3am, I do not guess. I replay 3am and look.

What does this catch that a status field never could?

Orphaned runs. A task that started, crashed, and left no finish line. The status field had already moved on. The event log still had the dangling start, so the check flagged it.

Leaked locks. A run grabbed a lock, died, and never released it. The next run blocked forever and looked "pending," not failed. The acquire-with-no-release pattern made it obvious.

Parity gaps. The finish-only ledger and the event stream should agree on how many runs happened. When they disagree, one of them is missing reality. The log is the tiebreaker because it cannot be rewritten.

Start small

You do not need a queue, a message broker, or a new dependency. A flat JSONL file and an append are enough to begin. Emit start and finish for your runs. Add lock events when you have shared resources. Write a tiny reader that flags any start without a finish.

The discipline that makes it work is the append-only rule. The second you let code edit old events, you are back to a status field with extra steps. Keep it immutable and the log will tell you the truth your dashboard hides.

If you want runtime limits on top of that visibility, AgentGuard caps token, budget, and rate spend per agent so a stuck loop stops before it drains your account: https://bmdpat.com/tools/agentguard

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

A self-healing system can't heal an empty queue

Patrick Hughes — Sun, 21 Jun 2026 14:45:09 +0000

A self-healing system can't heal an empty queue

My blog pipeline went red two mornings in a row. The self-healing step ran both times and fixed nothing. The bug was not in the healer. The bug was that I asked it to heal the wrong kind of failure.

Short version: automated recovery only works when the failure is "the machine broke." When the real failure is "the machine has no work," retrying does nothing, because there is nothing to retry against. A monitoring check that flips red has to tell those two cases apart, because the recovery for each is the opposite of the other.

What actually broke

I run a small fleet of scheduled tasks that publish one blog post a day. The chain is simple. A draft lands in a queue. A reviewer checks it at 08:30. A publisher ships the approved one at 09:30. A separate check runs after that and asks one question: did a post go live today?

If the answer is no, the check flips red and calls a healer. The healer's job is to get a post live. For weeks it worked. Then two mornings in a row it ran, reported failure, and left the dashboard red.

I went looking for the bug in the healer. There wasn't one. The healer did exactly what it was told. It looked for an approved draft to publish, found none, and correctly reported that it could not publish nothing.

The queue was empty. No draft had been written. The publish step had nothing to ship, and no amount of retrying an empty queue produces a post.

Two failures that look identical on a dashboard

Here is the trap. On the dashboard, both of these render as the same red box: "no post went live today."

The machine broke. A draft existed, but the publish step crashed, hit a bad credential, or got a 500 from the API. Recovery: retry. Fix the bug, run it again, the post ships.
The machine had no work. No draft existed. Recovery: retrying does nothing forever. You have to manufacture the missing input first.

These are opposite problems. The first says the input was fine and the action failed. The second says the action was fine and the input was missing. A healer built for the first is useless against the second, and most self-healing automation only handles the first. It assumes the work exists and the step failed. When the work itself is missing, it spins.

How I split the healer

The fix was to stop treating "no post today" as one failure. It is two.

When the check goes red, it now asks a second question before doing anything: is there content waiting? If a draft exists and the publish failed, that is a broken-machine problem, and retry is the right move. If no draft exists, retry is pointless. The recovery for an empty queue is not to republish. It is to generate the missing draft.

So the empty-queue healer is a content generator, not a republisher. It writes a fresh post, runs it through the same quality gate every other post clears, and only then publishes. This post is the output of exactly that path.

Doesn't auto-generating content just make filler?

That is the real risk, and it is worth naming. The moment your recovery for "no work" is "manufacture work," you have built a machine that can publish garbage to make a red light turn green.

The guard is the quality gate. The generator can write a draft, but it cannot publish one that has not passed the same independent review a queued draft passes. If the generated draft fails review, the pipeline stays red and asks me for a decision. Red is honest. A green light earned by shipping filler is worse than an honest red.

That is the line. A healer may manufacture the missing input, but it may never lower the bar that input has to clear. Generate freely, publish only what passes.

Why this matters past blogging

Any check that flips red on a missing outcome has this fork in it. Did the nightly report not run because the job crashed, or because there was nothing to report? Did the deploy not happen because it failed, or because there was no new commit? Retry fixes the first. It is a no-op on the second.

Before you wire up automated recovery, write down which failure you are recovering from. If you only handle "the step crashed," your healer will sit there retrying an empty queue and calling it an outage.

Where the cost angle comes in

A healer that retries the wrong thing is annoying when the step is a publish call. It gets expensive when the step is a paid model call. A retry loop pointed at a failing API spends real money on every attempt, and an agent that retries on a schedule can do that all night while you sleep.

That is what I built AgentGuard for: a budget, token, and rate limiter you wrap around an agent so a runaway or retry loop stops at a hard ceiling instead of grinding until the invoice teaches you. Free to install: pip install agentguard47.

When your monitor goes red tomorrow, ask the second question before you retry: is this broken, or is it empty? The answer decides whether retrying helps or just burns time and money.

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Missing AI agent cost data is not zero

Patrick Hughes — Fri, 19 Jun 2026 14:45:08 +0000

Missing AI agent cost data is not zero

My agent spend ledger showed $0 for the day. The agents had run all morning. The number was a lie, and the bug behind it is one almost every cost tracker ships with.

Short version: when a provider's billing data has not arrived yet, a naive cost tracker records $0 for that period. For AI agents that run unattended, this hides the exact spending you built the tracker to catch. The fix is to model missing data as a distinct "unknown" state, never as zero, so a day you cannot measure reads as unmeasured instead of free.

Why a $0 day is the dangerous one

I run a fleet of scheduled agents. They digest articles, review drafts, sweep a queue overnight. Real model calls, real dollars. I built a daily spend ledger to answer one question every morning: what did yesterday cost, and am I about to blow the budget?

The first version summed whatever cost data it had and printed a total. Most mornings it printed a small number. Some mornings it printed $0.

The $0 mornings were not free mornings. They were mornings where the provider billing data had not reported yet. Usage lags. Some providers only give you a CSV export you pull later. So at 07:00 when the ledger compiles, the cost of the run that finished at 05:30 often is not available.

The ledger had no way to say "I don't know yet." It only knew how to add. No data plus no data equals zero. So it reported zero, looked green, and moved on.

The failure hides exactly what you were watching for

Here is why this is worse than a normal rounding error. Agents run while you sleep. A retry loop, a runaway tool call, a prompt that balloons context: these spend money at 3am with nobody watching. That unattended spend is the whole reason a budget tracker exists.

Billing lag and runaway spend show up at the same time. The surprise charge arrives late from the provider, which means the day it happened is precisely the day your ledger had no data for. A tracker that reads missing data as $0 fails in the one situation you built it for. It tells you everything is fine on the days you most need a warning.

Model three states, not one number

The fix is small, and it is a data-modeling fix, not a math fix. A cost period has three states, not one:

State	Meaning	Ledger shows
known	provider data is in	the real number
partial	some sources reported, others pending	the partial sum, flagged
unknown	no data has arrived yet	"unmeasured", not $0

A day with no billing data is unknown. The report says so in plain words. It does not get to pose as a cheap day. When the provider export lands, the day flips to known and the real cost backfills in.

This sounds obvious written down. It is the same null-versus-zero bug that has burned every database schema since forever. Zero is a measurement. Missing is the absence of one. Collapsing them throws away the single most important fact: whether you actually know.

What this looks like in practice

My ledger now writes one of those three states per source per day. If an export has not been pulled, that line says unknown, and the daily total carries an "incomplete" marker until every source reports. I would rather see "we cannot confirm yesterday's spend" than a confident, wrong $0.

The rule I wrote down for every agent that touches the ledger: missing provider billing data is not zero. Mark it unknown until a real export or API confirms the number.

That one line stops the green-dashboard trap. A system that defaults to zero when it is blind will always look healthy in the moment right before the bill teaches you otherwise.

Stop the spend, do not just measure it

The ledger tells you what a run cost after the fact. It does not stop a run from spending the money in the first place. For that you want a hard cap on the agent itself.

That is what I built AgentGuard for: a budget, token, and rate limiter you wrap around an agent so a runaway run stops at a ceiling instead of grinding until the invoice surprises you. It does not care whether your billing data has arrived. It counts spend as it happens and pulls the plug at the number you set. Free to install: pip install agentguard47.

What does your cost tracker show on a day the billing data is late: a real unknown, or a comforting $0?

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Anthropic Writes 80% of Its Code with Claude

Patrick Hughes — Thu, 18 Jun 2026 14:45:11 +0000

What does 80% AI authored code mean for solo devs?

In June 2026, Anthropic stated that about 80% of its new production code is authored by Claude. When a major AI vendor hits that volume, the shift is undeniable. For a solo developer or a one-person holding company, this changes the math entirely. The bottleneck is no longer typing characters. The bottleneck is review and ownership.

When you run a solo shop, you do not have a team to absorb the review burden. If your agents write 80% of the code, you still have to read, understand, and answer for 100% of it. AI writes the code, but you still own the outcome.

How do you manage the review burden?

The only way to survive high-volume AI output is to build verifiable constraints. You must prove the code works without reading every line.

In my own vault, nightshift agents ship PRs while I sleep. They run a multi-step loop. They write the plan, build the tests first, write the code, and then spawn an independent QA subagent to review the diff. The QA agent acts as the gatekeeper. It fails the PR if rules are broken. It flags secrets and checks constraints. I wake up, review the clean PRs, and merge them.

You must enforce boundaries. If an agent drifts into forbidden paths or tries to merge a broken build, the system must auto-revert or block it.

Who answers for the code?

Andreas Kling of the Ladybird browser project asked a vital question. Who answers for the code? When AI writes the bulk of your logic, the human reviewer is the final backstop.

Self-reported numbers like 80% measure volume, not quality. Volume is easy. Correctness is hard. You cannot blindly trust the output. You must verify it mechanically.

If you are a team of one, your agents are your peers. You need guardrails on what they can do. If you want to put a runtime budget on your agents, check out AgentGuard.

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

What Salesforce's 20,000 AI Agent Deployments Teach a Solo Builder

Patrick Hughes — Thu, 18 Jun 2026 14:45:08 +0000

Salesforce has shipped around 20,000 Agentforce deployments. ByteByteGo published a writeup of what they learned, sourced to John Kucera, the CPO of Agentforce. I run a one-person agent fleet, which is about as far from Salesforce scale as you can get. The lessons still translate. Better than I expected, actually.

Short version: 90% of agent work happens after launch, not before. The failures cluster into three patterns. Putting deterministic logic inside an LLM loop, prompting harder instead of encoding policy in code, and feeding the model way too much context. All three are engineering problems, not model problems.

Why does 90% of agent work happen after launch?

Traditional software front-loads the effort. You spec, build, test, then go live and mostly maintain. Agents invert that. Modern tooling gets you a functional demo in hours, and that speed creates false confidence. The demo covers the typical cases. Production brings edge cases, ambiguous phrasing, and questions that cross domains your agent never saw in testing.

I have lived a small version of this. Every agent I run looked done on day one. The real work was the weeks after: the input that arrived in a format I never tested, the API that returned something half-empty, the task that technically succeeded while producing nothing useful. If you budget your effort assuming launch is the finish line, you will abandon the agent right when the actual work starts.

Salesforce's advice here is blunt: do not boil the ocean. Start with one narrow, high-value use case so your iteration cycles stay fast. At solo scale that means one agent, one job, one queue. Get it boring before you add the second one.

What are the three anti-patterns that degrade agents?

First: over-reasoning deterministic workflows. If you can flowchart the logic, it belongs in code. Salesforce built Agent Script, a TypeScript framework that mixes deterministic control flow with LLM reasoning, because asking a model to re-derive an if-else chain on every run is slow, expensive, and occasionally wrong. You do not need their framework. You need the rule: flowchart it, then script it. Save the model for the parts that are genuinely ambiguous.

Second: prompting harder instead of encoding policies. Writing NEVER and ALWAYS in caps does not reliably constrain a model. Salesforce found business rules have to execute independently of model reasoning. This one matters most for small shops, because prompting harder is free and feels like progress. If a rule actually matters, enforce it in code that runs whether or not the model cooperates. A refund cap belongs in the payment function, not in paragraph four of the system prompt.

Third: poor context engineering. One e-commerce team in the writeup cut an order API response from 100K tokens to 2K by returning only the relevant fields. The agent got faster and more accurate at the same time. That is the detail worth tattooing somewhere: less context made it better, not just cheaper. Dumping a whole API response into the prompt is the default, and the default is wrong.

How do you know an agent is actually working?

Salesforce measures Agentic Work Units, meaning actual task completion. For support agents they track containment rate: cases resolved without human follow-up. Outcomes, not activity.

I learned a version of this the hard way. A scheduled agent can exit zero every night and produce nothing. Green checks lie. The fix is to check the declared output, not the exit code. Did the file appear, did the post go live, did the ticket close. Whatever your equivalent of containment rate is, measure that.

Their post-launch triage is also worth stealing. Issues get split four ways: tone or brand drift means fix the prompts, logic errors mean fix the tools or convert that step to a script, data quality problems get routed to whoever owns the source, and coverage gaps mean expand scope or escalate cleanly. Four buckets, four different fixes. Most solo builders treat every failure as a prompt problem. Most failures are not.

What does this mean if you're not Salesforce?

Salesforce has platform teams to absorb the post-launch 90%. You have you. That changes the build order, not the lessons.

Move deterministic logic out of the loop first. It is the cheapest win: fewer tokens, fewer surprises, faster runs. Then encode your real rules as code-level checks the model cannot talk its way past. Then cut your context down to what the task needs. Each of these makes the after-launch grind smaller, which at solo scale is the difference between a fleet you maintain and a fleet that quietly rots.

And put hard runtime limits on every agent before it touches production. The deployments in the writeup degrade in ways nobody predicted in the demo, and at 20,000 deployments Salesforce can eat the bad days. One runaway retry loop on your side is your whole margin. That is the exact surface I built AgentGuard for: per-agent budget caps, token limits, and rate limits enforced at runtime, not in the prompt. It is a pip install, agentguard, and it takes minutes to wire in. Start there: https://bmdpat.com/tools/agentguard

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

57-71% of AI agents leak data between users. Here's what to do.

Patrick Hughes — Wed, 17 Jun 2026 14:50:11 +0000

57-71% of AI agents leak data between users. Here's what to do.

Summary: A June 2026 Mem0 survey reveals that 57-71% of agent harnesses leak memory between users. This happens because most systems use keyword retrieval without user isolation. Builders must implement per-user namespaces and principal checks to prevent PII leaks and credential bleed.

Mem0's June 2026 survey of 8 major agent harnesses included Claude Code, Codex, and Bedrock AgentCore. They found a 57-71% cross-user memory contamination rate. Most of these systems rely on keyword retrieval. They lack user-scoped isolation.

If you run agents for multiple users, your memory layer is likely leaking.

Why does keyword retrieval fail across users?

Most agent runtimes use simple keyword matches to pull relevant memories into the context window. This works well for single-user assistants. It fails in multi-user environments because the retrieval layer has no concept of a principal.

When User B asks a question, a fuzzy match might pull a memory fragment written by User A. If User A stored PII or credentials, those secrets are now in User B's prompt. The agent has no way to know it just crossed a security boundary.

What are the real failure modes of memory contamination?

Memory contamination is not just a style issue. It creates three critical risks:

PII leak: Personal data from one user appears in another's session.
Decision contamination: A policy or preference set by User A influences the agent's actions for User B.
Credential bleed: API keys or tokens stored in memory by an admin become accessible to a standard user.

How do you fix the agent memory layer?

To build secure multi-user agents, you need to move beyond simple keyword search. Use these four patterns:

1. Per-user namespaces. Every memory must be tagged with a unique UserID. The retrieval query must include a hard filter on that ID.

2. Recall-time principal checks. Before a retrieved memory is injected into the prompt, verify that the current session principal has read access to that specific memory object.

3. TTL and staleness handling. Memories should not live forever. Implement time-to-live (TTL) settings and session-based eviction to ensure sensitive data does not linger in the vector store.

4. Vector partitioning. Use physical or logical partitioning in your vector database to ensure that a search for one user cannot even "see" the data of another.

How does AgentGuard help secure agent memory?

Isolating memory is only half the battle. You also need to enforce scope at the action layer.

AgentGuard provides the runtime budget and scope enforcement that acts as the action-layer analogue of memory isolation. Just as you should not let an agent recall User A's data for User B, you should not let an agent spend User A's budget on User B's tasks.

By wrapping your agent in AgentGuard, you ensure that even if a memory leak occurs, the agent's ability to act on that leaked data is strictly bounded by the current session's security policy.

Learn how to secure your agent runtime with AgentGuard.

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

VRAM Calculator: Estimate Local LLM Requirements

Patrick Hughes — Mon, 15 Jun 2026 14:45:10 +0000

What is the VRAM Calculator?

Running local LLMs requires knowing your hardware limits. I built the VRAM Calculator to help you estimate the video memory needed to run models like Llama 3 and Mistral. Knowing your constraints before downloading a 40GB model saves you hours of frustration.

The Math Behind It

Estimating VRAM is more than just checking the base file size. You have to account for context window length, quantization levels like GGUF Q4 or Q8, and inference engine overhead. The calculator handles the math and gives you a concrete target for your setup.

How It Compares

Static reference tables get outdated fast. This calculator uses dynamic estimates based on real memory footprint data from local AI engines like llama.cpp.

You can use the tool right now: Try the VRAM Calculator.

Ready for Production?

If you are deploying AI agents and need to monitor their execution safely, check out AgentGuard.

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.

Anthropic's IPO and the 40% Cost-Savings Gap: Why Your Spend Cap Matters More Now

Patrick Hughes — Sun, 14 Jun 2026 14:45:11 +0000

Anthropic filed confidentially for an IPO. Two newsletter bullets I read on 2026-06-04 (TLDR AI and FutureTools) put the post-money valuation at $965B after a $65B Series H raise. Revenue run-rate is reported at $47B, up from $9B at the end of 2025.

Here is the part that should get your attention as a builder. The same bullets report that 40% of enterprise customers say they got under 10% cost savings from their Claude deployments.

Read those two numbers together. Revenue is 5x in about six months. And almost half of enterprise buyers say the value is not showing up in their bills. That is the exact shape of a re-pricing event.

Why the 40% gap exists

The gap is not about the model being slow or wrong. It is an accounting mismatch.

You pay per invocation. You get value per completed goal. Those are not the same thing, and the difference is where your money leaks.

Three places the tokens go to die:

Retries. A tool call fails, the agent tries again, then again. Each attempt bills. None of them shipped the result.

Dead-end branches. An agent explores a plan, burns tokens, then abandons it. You paid for the exploration even though nothing reached the user.

Unverified completions. The agent says "done." Nobody checked. You paid full price for an output that was never confirmed to be correct.

None of this shows up as a single scary line item. It shows up as a bill that is bigger than the work you can point to. That is the 40% gap in one sentence.

What the IPO changes for your team

A confidential filing means public-market disclosure is coming. Public markets reward margin. The cheapest way for any vendor to defend margin is to adjust pricing on the tiers that are currently subsidized.

I am not predicting a specific price hike. I am saying the incentive is now pointed in one direction. If you are running production agents, plan for the cost of a token to matter more next quarter than it did last quarter.

The wrong move is to panic-switch vendors. Migrating an agent stack is expensive, and the next vendor has the same per-invocation-versus-per-goal problem. Switching does not fix the leak. It just moves it.

The right move is to cap the spend and verify the goal before you pay for it. Keep a real exit option open too. Running a small local model on consumer hardware is a credible fallback for some workloads. I wrote about that in local LLM inference on consumer GPUs.

The pattern: budget per goal, hard stop, audit trail

This is the gap AgentGuard was built for. It is a runtime budget limiter for AI agents. You set a budget per goal, a cap per key, a hard stop, and you get an audit trail of where the tokens actually went.

from agentguard import Guard

guard = Guard(budget_usd=0.50, per_key_limit=100_000)

with guard.track(goal="summarize-ticket"):
    result = run_agent(ticket)
    guard.verify(result)  # only counts as paid value if the goal check passes

The point is not the exact API. The point is the shape. You declare what one completed goal is worth before the agent starts. The agent runs under a hard ceiling. When it hits the cap, it stops instead of quietly burning another dollar on a dead-end branch. And the audit trail tells you which goals actually completed, so you can see the 40% gap in your own numbers instead of guessing.

That last part matters most. You cannot manage a leak you cannot measure. Per-goal accounting turns "the bill feels high" into "these three goals burned 60% of spend and only one of them shipped."

Get ahead of the re-pricing

The news moment is the IPO. The durable lesson is older than this filing. Per-invocation pricing and per-goal value will always drift apart, and that drift is your cost problem.

If you want the deeper version of this, I keep a hub post on AI agent cost and pricing and a hands-on walkthrough of cost control with AgentGuard in Python.

Cap the spend. Verify the goal. Pay for value, not for retries. Start with a budget cap before the next pricing event lands: https://bmdpat.com/tools/agentguard

Originally published on bmdpat.com. I run a one-person AI agent company and write about what actually works.

Want these in your inbox? Subscribe to the newsletter - no spam, unsubscribe anytime.