DEV Community: Fabian Both

The AI Automation Landscape in Four Quadrants (and the One Nobody Has Solved)

Fabian Both — Tue, 07 Jul 2026 11:58:11 +0000

Many people are automating with AI right now, and the tooling landscape has become confusing. Chat assistants with connectors, skills, schedulers, workflow builders with LLM steps, agent platforms, vertical suites. They all claim to automate your work, and they are all sort of right, for different kinds of work.

Here is the map I use to sort this out. One scoping note: I am leaving coding out, it is its own landscape with its own verification story and deserves its own post. Everything below is about business and knowledge work.

Quadrant 2: unattended + deterministic (repetition without judgment)

n8n, Zapier, Make. These run while you sleep, and they are reliable precisely because they are rigid: when X happens, do Y, no judgment involved, nothing to get wrong except the plumbing. If your task is truly deterministic, this is the correct quadrant and an agent would only add cost and variance. The trouble starts when the task needs actual judgment and the workaround is an LLM step in the middle of the workflow. You get a deterministic pipe with a stochastic segment: the pipe cannot reason about what the LLM produced, and the LLM cannot recover when the pipe feeds it something unexpected.

In practice this setup is how a lot of unattended agentic work gets done today, not because it is the right tool but because it is the tool people already have. I did the same, and the failure mode was always the same: the run is green. One of my own monitoring jobs finished cleanly, seven rows written, no errors, and a review afterward found an ignored filter rule, a duplicated entry, and three plausible-looking values that were simply wrong. The engine reports success because it checks that steps executed, not whether the judgment in the middle was any good. I wrote that story up here.

Quadrant 3: attended + agentic (one-off work with judgment)

This is where most people experience AI automation today. You sit in a chat with Claude or another assistant, you have your calendar, mail, and docs connected, and you delegate: search this, draft that, pull the context from those three places and summarize it. Skills let you package recurring shapes of tasks so you do not have to re-explain them every time.

I use this constantly, and I want to be clear: this quadrant does not need to change. One-off tasks are naturally attended. You steer mid-task, you catch the wrong recipient before a draft goes out, you notice when the agent pulled the wrong document. Errors die young because you are watching. The reliability of this quadrant is not a property of the agent. It is a property of you sitting there.

That is not a flaw, it is the design. The only limit worth naming is that attended automation saves effort but not attention. It cannot run at 7am without you.

Quadrant 4: unattended + agentic (recurring work with judgment)

The youngest quadrant, and it exists because a real category of work exists: tasks that repeat or should happen proactively, and that need judgment each time. Monitoring things and deciding what matters, triaging what came in overnight, preparing work before you ask for it. Too repetitive to do attended, too judgment-heavy to do deterministically.

Several kinds of tools are trying to serve this category, and their origins show. The chat providers extend their assistants with schedulers, today mostly for read-only work like briefings. Independent agent frameworks like OpenClaw and Hermes run unattended with real access, often with more enthusiasm than safeguards. The workflow builders add agent products on top, Lindy, Zapier Agents and similar, where you state a goal, grant access, and set a schedule. The public reporting is consistent: short chains work, longer chains drift, and the recurring unattended runs are where users report getting burned.

And the vertical suites build the whole thing for one narrow domain. I have first-hand experience there: at Octomind we built exactly this for website testing, and what made it work was not a smarter agent but the dedicated harness around it, verification built into the domain plus human interaction points to surface and control the system.

Agents ship with capability. Safety and reliability have to be built around them.

That is the pattern underneath quadrant 4, and it is the sentence this post exists for.

An agent with tools is pure capability. Models keep getting better, connectors keep multiplying, and none of that is the bottleneck anymore. What is missing is the counterpart: the part that faces the real world the agent acts in and makes the whole thing trustworthy. When I break down what "I can trust this agent unattended" actually requires, I get three parts:

Safe. The agent cannot cause damage outside its mandate: scoped tools and permissions, sandboxed infrastructure, credentials that cannot leak, resistance to prompt injection. Zapier for example ships PII redaction and injection detection. This part is being worked on across the industry.
Reliable execution. No run is silently swallowed: retries, durable state, surviving API timeouts. This is hard engineering, and plenty of tools still get it wrong today, n8n included. But it is solvable with known patterns, and the serious players are investing here. Lindy for example moved its agent execution onto Temporal for exactly this.
Reliable AI. The run did what it should have done, and what it claimed to have finished is actually finished. Not "did it execute", not "was it contained", but "was it right, this time".

The industry is investing in safety and execution and mostly skipping the last part. That is understandable: the first two are hard but conventional engineering problems, with known patterns and a clear definition of done. Reliable AI has neither. It requires judgment about a specific run in a specific context, which is exactly what you cannot get from retries and filters. Your platform can guarantee that your morning brief job ran at 7am. Nothing checks whether the brief was wrong.

This is the capability-reliability gap from earlier, seen from the tooling side. Quadrant 4 is a legitimate category with real demand, and it is currently served mostly by tools that brought their capability along and left the counterpart behind.

How to make unattended agent work trustworthy

If reliable AI cannot come from retries and filters, it has to come from supervision: something independent that looks at each run and judges the outcome. The obvious objection: if AI reviews AI, why trust the reviewer? Because you do not ask the reviewer to be right, you ask it to classify and escalate with a safety margin. Flagging a run as "a human should look at this" is a much easier problem than doing the task correctly in the first place. The catch is false positives: a noisy signal is one nobody reads, so escalate too often and people tune it out, and you are back to trusting runs blind. The real work is driving false positives down without narrowing the margin so far that real failures slip through.

That balance is reachable with a static reviewer today: a well-tuned one already makes unattended runs something you can trust, no learning required. Feedback is how it gets better over time, as a recurring task raises the same escalations over and over and those resolutions tune the reviewer, and eventually the agent, to stop flagging what keeps turning out fine. That learning loop is where this is heading, and it is the hard part: keeping a reviewer that tunes itself from drifting off course is a problem no one has solved yet.

Golemry is my attempt to build this as infrastructure: recurring agent jobs where an independent overseer reviews every run and escalates what needs human attention, available to any agent rather than tied to one framework or one vertical. There may be others working on this layer, and honestly, there should be.

The takeaway

If a task is one-off and needs judgment, do it attended, that works today. If it is truly deterministic repetition, use a workflow builder, that works too. And if it repeats and needs judgment, you are in quadrant 4, and before you trust any tool there, ask what it does after the run. If you are choosing between concrete options for a recurring task today, I compared them here.

Quadrant 4 is not solved by more capable agents. It is solved by what you build around them.

Why AI Automations Fail: The Capability-Reliability Gap

Fabian Both — Tue, 23 Jun 2026 08:50:20 +0000

An AI automation works on day one. The output matches what you pictured, so you ship it, schedule it, and move on. A few weeks later it's worse, and nothing told you. No error, no alert, no failed run. You only notice because you happened to look.

If you run any kind of recurring AI job, you've probably lived some version of this. It isn't bad luck or a weak prompt. It's structural, and it has a name worth understanding before you hand an automation anything that matters.

Why AI fails silently, and traditional software doesn't

Traditional software is deterministic. Same input, same output. When it breaks, it breaks loudly: an error, a crash, a timeout. You know something is wrong because the system tells you.

AI doesn't work that way. A language model produces confident, well-formatted output whether the answer is solid or invented. There's no built-in uncertainty signal, no warning light, just text that looks exactly like good text. So the failure mode isn't a crash. It's a fluent, plausible answer that happens to be wrong, and nothing flags it until it's already been sent, published, or acted on.

If you come from a world where software either works or it doesn't, this is a new thing to operate around. It's the same silent failure a green run can hide inside a workflow builder, and it changes how you have to run AI in production.

The capability-reliability gap: what the data shows

The gap between what AI can do and how reliably it does it is widening, not closing. When Princeton researchers profiled fifteen agentic models across two benchmarks in early 2026, capability had climbed steadily while reliability barely moved (the paper, or Fortune's writeup). Models that scored far higher on benchmarks stayed inconsistent and prone to silent failure in practice.

Doing something impressively once and doing it reliably a thousand times are different problems, and the industry optimizes the first, because that's how model providers compete. Reliability lags because it has to: you can't make something reliable before it's capable. Which means everyone adopting AI automation right now inherits the hard part, making it work consistently, almost entirely themselves.

This isn't only academic. Amazon's retail site saw repeated outages in March 2026, with internal documents reportedly pointing to AI-assisted code changes as a factor, and the fix was more human oversight, not less: senior engineers now sign off on junior engineers' AI-assisted changes. The broader numbers rhyme. RAND has estimated the AI project failure rate at over 80%. These aren't teams that didn't try hard enough. They're teams that built something, watched it degrade, and couldn't make it stick.

What vibe coding already taught us

Developers went through this exact transition with AI coding tools. The first wave was vibe coding: describe it, let the AI write it, ship without looking too closely. Fast, exciting, and it produced code nobody could maintain. The correction wasn't to abandon AI. It was to put review back in as a default step between generation and delivery. The bottleneck moved from writing code to reviewing it.

The same shift is now happening everywhere AI runs unattended: marketing, operations, research, support. The AI does the work. Someone still has to check it. Right now that someone is you, by hand, for every output, with nothing in place to make it better over time.

The three ways the gap shows up

The gap isn't one failure. It wears a few faces, and each has its own deeper write-up:

It degrades. A job that worked on day one quietly gets worse as the world changes and the prompt doesn't. Why your AI agent's output gets worse over time.
It doesn't survive scale. One AI cron job is easy. The tenth is where the cracks show. From cron job to production automation.
It hides behind a green run. A workflow reports success while the output is wrong. The silent errors workflow builders miss.

Different entry points, same root cause: a non-deterministic system running unwatched, with nothing able to tell a good run from a bad one.

How to close the gap: the overseer

Manual review works when you run one or two automations. The moment you run ten or twenty, it falls apart. You can't read every output every day, so you spot-check, or you stop, or you hope. That's not a workflow. It's a liability.

The fix isn't to review everything forever. It's to make review a default step that something else performs, and to let your corrections improve future runs. We call that something the overseer. The overseer is a separate evaluator that reads each run, not just its result, and escalates only what needs a human.

Three properties make it work, and they're worth stating plainly:

Separate the doer from the checker. The agent that produced the output is the worst judge of whether it's right, for the same reason you don't review your own code. A dedicated evaluator catches drift the executor can't see in its own work.
Make review the default. Every run gets read before its output ships, not after something breaks.
Close the loop. When you correct a run, that correction tunes future runs, so the same mistake stops repeating and you review less over time.

This is the pattern we build into Golemry, and it's what the capability-reliability gap actually calls for. Not a smarter model. A second set of eyes that reads the run.

Agent Automations You Don't Have to Babysit

Fabian Both — Tue, 09 Jun 2026 07:55:06 +0000

A few months ago I was poking at OpenClaw on a throwaway side project, just to see what it could do. At some point I told it, in plain English, to go do a small recurring task for me. It went off, wrote a cron job, wired up the connector, and that was it.

I sat there for a second. The thing my last company had spent weeks building in n8n, the scheduling, the integrations, the glue between them, was just there. Because the agent already had connectors and could talk to a scheduler, the entire integration layer that made a tool like n8n valuable had quietly evaporated. You said what you wanted and it set itself up.

Then the second thought arrived, and it was the one that mattered. I would never let this run anything I actually cared about. Not unattended, and definitely not in a business.

That gap is the whole story. The setup problem is mostly solved. The trust problem is not. This post is about the piece that is missing, which I have started calling the overseer.

The no-code pipeline that wasn't

First, some credit where it is due. n8n solved a real problem, and it is still the right tool for plenty of jobs. Recurring work needs to run on a schedule, survive restarts, and keep going when you are not watching. That is durable execution, and it is genuinely hard. n8n made it approachable.

The trouble showed up at both ends, the authoring and the execution.

At my last company we needed to automate outreach. Pull prospects out of the CRM, decide who was inbound and who was outbound, draft a personalized message for each, enrich it with data from the web, and send the email. On paper this was going to be a clean, non-technical pipeline that a non-engineer could own.

It did not stay that way. We needed code to split and reshape the data streams. The LLM steps were heavily constrained, and the models kept returning output we could not use directly, so we had to define structured tool outputs and add parsing steps just to turn text back into objects the pipeline could process. On top of that the thing was flaky. The AI step would time out, or return nothing, or quietly misbehave. What was sold as a no-code workflow had become a very technical, very fragile process that someone had to keep nursing.

The reason the authoring was painful is the reason the visual-workflow model exists at all. You have to pre-specify every branch up front, because you cannot trust the runtime to improvise. The graph pins down behavior you do not trust to be flexible. A capable agent removes that cost. You describe the task once, in words, and it works out the steps.

But the execution problem does not disappear with it. Those tools guarantee durable execution for deterministic steps, and an LLM step is not deterministic. The moment one sits in the middle of the pipeline, the guarantees you were leaning on stop holding, which is why ours kept timing out and going quiet. So the authoring gets easier and the execution gets harder, and you are left needing a runtime built for non-deterministic work from the start.

What agents took, and what they left behind

Agent connectors took the integrator moat. The hard, defensible part of the old tools was the long list of integrations and the wiring between them. An agent with connectors and a scheduler does that for free, on demand, from a sentence.

What they did not give you is the ability to walk away.

A capable local agent will happily do broad work for you. But to run it unattended you have two options, and both are bad. You let it ask permission on every step, which means you are still sitting there, so nothing was actually offloaded. Or you let it run on its own and hope, which is fine for a toy and unthinkable for anything that matters. These agents were also never built to run in the cloud under a real permissions model, scoped to exactly the access a single job needs.

So the bottleneck moves. It is no longer setup, and it is no longer capability. The bottleneck is you, watching. Run one automation and you can keep an eye on it. Run ten or twenty and watching for the one that broke becomes a full-time job. You saturate, not because the agents cannot do the work, but because you cannot review all of it.

That is the part I wanted to solve.

What it takes to stop watching

The list of things that have to be true before you can leave a job alone is short and mostly boring. The job has to run when it should and fail loudly when it cannot. It has to be scoped to exactly the access it needs and sandboxed, so a job that goes wrong cannot wander into things it should never touch. And something has to watch each run and decide, every time, whether to escalate it to you or stay quiet. With deterministic code, success is a binary you can check: the test passed or it did not. With an agent there is no such hard criterion, so whether a run actually went well is a judgment, not a check. The first two are table stakes. The last one is the whole game. It is what makes review scale at all, separating the runs you can safely ignore from the few that actually need you.

And that watcher has to be separate from the worker. An agent grading its own work skews positive, which is why Anthropic's recent harness work pairs a generator with a separate evaluator instead of asking one agent to do both. The worker wants to finish. The checker has to want to find the problem.

So I built that watcher

It is called Golemry. It is the layer the local agent never had: infrastructure for the recurring work.

You add it to the agent you already use as an MCP server. You describe a recurring task in plain language, your agent sets up the job, and from then on the job runs on Golemry, on a schedule, scoped to its tools, sandboxed, with an overseer reviewing every run and pulling you in only when something looks off. Your agent stays the interface. It sets the jobs up, checks what ran, and hands you the one thing that needs you.

Here is the kind of thing it catches. I had a weekly research job that kept emailing me a normal-looking overview while the work behind it quietly went shallow. The output never gave it away, but the overseer reads the run and not just the result, so it caught the reasoning going thin and flagged the job as outdated. Nothing in the email would ever have told me.

Your agent, minus the babysitting

Picture the agent you use now, connectors and all, handling your daily back-and-forth. Now it grows one new ability. Anything recurring, the weekly research overview, the crawl for post ideas and drafts, the report you would otherwise check every morning, it can hand off to something built to be left alone and trusted to call you back.

That is the difference between automations that compound and automations that strangle you. You stop being the ceiling on how many can run at once. And because the running and the permissions live in the infrastructure rather than on your machine, you can finally point an agent at this in places you never would have before.

Building this in public

I would rather be precise than impressive, so here is the honest state of it.

Today, in V1: scheduled jobs set up through the MCP server from your agent, sandboxed runs, each job scoped to the tools it needs, a large library of connectors, and the overseer reviewing each run after the fact and escalating with a verdict by email and a notification.

Next: human-in-the-loop tooling, where the overseer weighs in on a proposed action before it goes out, and event-based triggers, so a job can fire on something happening rather than only on a clock.

Still open: the feedback loop and learning piece, where the overseer's verdicts and your responses become a threshold the job tunes over time, escalating a little less as a job earns trust and a little more when it slips. The crux here is how to expose this in an intuitive way and how to scope it.

The goal does not change at any stage. Move from watching everything to watching almost nothing, gradually, on guarantees you can point at instead of a feeling you hope holds.

V1 is live. You can try it at golemry.com, and the roadmap is where the human-in-the-loop and learning pieces are headed. Try it, and tell me where it breaks. That is the kind of feedback I am building this on.