Every week brings a new agent that can browse, code or plan. The question I keep coming back to is less glamorous: how do you know the thing did the right work, and what happens to the humans who are supposed to catch it when it didn't? Over the past month I saved seven notes that circle that question from different directions: memory that learns reusable routines, rewards that grade each reasoning step, a language that refuses code that breaks stated rules, a 40-year-old paper on automation, and a scheduling method that treats estimates as evidence. They aren't a unified theory, but read together they make a decent checklist.
Agents that learn routines and get graded step by step
Agent Workflow Memory (AWM). The idea behind AWM is that a web agent shouldn't solve every task from scratch. It induces "workflows", common sub-routines with the example-specific details stripped out, stores them in memory, and reuses them on later tasks. It runs offline, where it learns from annotated examples, or online, where it builds workflows from its own past attempts with no extra data. The authors report a 35.6% success rate on WebArena and the top text-only score on Mind2Web. I wrote up how AWM's offline and online modes differ, along with the pipeline commands for each benchmark.
TRL's PRM Trainer. Outcome-only rewards tell a model whether the final answer was right; a process reward model (PRM) scores each intermediate step. The original paper behind it found outcome supervision gets you a similar final-answer error rate with fewer labels, but if you care whether the reasoning itself is correct, you need step-level feedback: reasoning errors among correct answers dropped from 14.0% to 3.4%. Hugging Face's TRL ships an experimental PRMTrainer that expects a dataset with prompt, a list of completion steps and a per-step label. My walkthrough of training a step-level reward model with TRL covers the Qwen2-0.5B + Math Shepherd example and the config knobs worth knowing.
Making "don't break this" checkable
Bend. Bend pitches itself as C-like speed, CUDA-like parallelism and Lean-style proofs behind Python-flavoured syntax. The part that matters for agents is LAWS.bend: you declare rules that must hold, and the type checker, which is really a proof checker, rejects any change that breaks them, so an AI edit that introduces a bug gets bounced until the law is proven again. The project describes it as an AGENTS.md backed by proofs, and claims checks fast enough (around a second on a mid-sized codebase) to run after every edit. It's still evolving and only recommended for backends, but Bend's proof-checked approach to AI coding is the most concrete answer I've seen to "how do I stop the agent from regressing this".
The human side of automation
Ironies of Automation. Lisanne Bainbridge's paper, cited more than 1,800 times, is the one I'd hand anyone building agents for operators. Its core irony: automation still needs someone to watch it and someone to take over fast when it fails, yet the more you automate, the less practice that person gets. More alarms can make an emergency more confusing, and people are asked to spot machine errors they can't judge in real time. Her suggestions are refreshingly practical: stop, observe, understand, correct, restart; pace automation to what humans can follow; keep operators trained with manual runs or simulators. Here's my summary of the Ironies of Automation paper, based on a thread that walks through it and Bainbridge's later revision.
Evidence Based Scheduling. Joel Spolsky's method isn't about AI at all, but it's the best antidote I know to single-number promises. Break work into tasks of 16 hours or less, log estimated versus actual time, turn that into a per-person velocity, then run a Monte Carlo simulation to get a probability curve for the ship date instead of one date. Interruptions get absorbed by the historical data rather than estimated separately, and you can watch the curve move as you cut low-priority features. If you're forecasting work that agents and people share, EBS and its Monte Carlo ship-date curve is worth a fresh look.
Finding the evidence and the context
alphaXiv. When I need to check whether a claim about agents holds up, I start from the papers. alphaXiv connects papers, researchers and institutions and is built around answering questions grounded in the underlying research, with an MCP server and a browser extension on top. The snapshot I saved listed work from late September and early October 2026 on context-editing language models, agents recovering from their own critical mistakes, and benchmarks for whether agents can design libraries for other agents. My note on using alphaXiv to keep up with agent research lists the sections that were most useful.
Conare. Plenty of agent failures aren't reasoning failures, they're missing-context failures. Conare sells context infrastructure for AI companies: one API that pulls product data, user history and past decisions from places like Gmail, Slack, Notion and Linear, so a model's answer cites where it came from. It's a young product and my note is mostly about its positioning, but Conare's single context API is a useful reference point if you're designing your own retrieval layer.
What I take from it
Verification shows up at every layer: memory that keeps proven routines, rewards that check steps rather than endings, compilers that check promises, schedules that check estimates against history. Bainbridge's warning sits over all of it. The better the checks get, the more deliberately you have to keep the human in practice.
These are my own reading notes; the projects named above belong to their respective authors.
Top comments (0)