DEV Community: Motty Chen

AI writes the code fast. The expensive question is what happens after something fails

Motty Chen — Wed, 21 Oct 2026 02:30:42 +0000

AI coding agents can implement a change ridiculously fast.

That's cool.

But the party is not over yet.

In my own IssueFlow workflow, implementation is only one stop. A normal production path can still include validation, security review, code review, UI/UX review, documentation, merge, merge validation, E2E against the deployed code, and cleanup.

And before implementation, there may already have been design, design review, and a human design approval.

This was not imposed on me. I built it this way on purpose. Fast code is nice. Code that is actually what I meant, fits the architecture, follows the standards, survives the real product, deploys correctly, follow my crazy quality requirements, and does not create the next fire is nicer.

The problem is what happens when one of those stages turns red.

“It failed” is not enough information

Take merge.
Thee are millions (almost) reason it can fail:

A merge can fail because there is a real conflict.
It can fail because CI is red.
It can fail because a required check never ran.
It can fail because GitHub is rate limiting us, a runner is having a bad day, or the infrastructure is unavailable.

Those are very different problems.

IssueFlow already had a failure-reason vocabulary before this week. That lets me say things like:

merge conflict → resolve the conflict (back to implementation)
real validation failure → rework the code (back to implementation)
checks timeout → retry the merge
infrastructure fault → wait/retry instead of rewriting code
design no longer fits → go back to design.

That part is useful, but it is not the change I am excited about.

The harder question comes after recovery.

The harder question comes after recovery

Suppose a merge conflict sends the issue to implementation for conflict resolution.

Great. The conflict is fixed.

Now what?

Do I send it through validation again?
Security review?
Code review?
UI/UX review?
Every single stage?

Well, sometimes the answer is yes.

But! Sometimes that is just burning time and tokens to prove something we already know. And that is where it hits my sleep time and my pocket.

The same thing happens with a security finding.

Imagine security review says a dependency needs to be updated. The agent updates the library.

Do I need to re-check product intent?
Probably not if the product behavior did not change.

Do I need security and affected tests again?
Absolutely.

Or maybe the rework touched only test code. Does that invalidate the UI review that already passed? Maybe not.

A failure reason tells me why the stage stopped. It does not necessarily tell me which parts of the remaining workflow still add evidence.

Static workflows know a lot at the beginning

When an issue enters IssueFlow, the workflow can route based on things like complexity, issue type, environment, priority, role, and other dimensions.

That is already useful.

A chore does not need the same path as a large feature. An epic child does not need the same entry path as a standalone issue. A production lane can demand more validation than another lane and deployment is different.

But software has an annoying habit of teaching us things after we start. I know this problem since the 90's...

Implementation may discover that the approved design no longer fits the current codebase.
A backend-looking change may turn out to affect the UI.
A security fix may change only a dependency.
A rework pass may touch only tests.
A merge problem may turn out to be nothing more than GitHub having a bad afternoon.
Or an issue may simply have gone stale because the product moved on.

Those facts do not exist when the issue first enters the workflow.
But the agent that just handled that stage - knows!

Let the workflow learn while it runs

I recently added a mechanism to IssueFlow that lets a stage set or change a routing dimension while the issue is already running.
That new dimension becomes part of the routing state for the issue.

Later routing decisions can use it.

That sounds like a small detail, but it changes how I think about recovery.

The stage is not inventing a new workflow. The allowed dimensions, values, routes, and stages are still defined by the workflow owner.

The difference is that the workflow can now update its understanding of the work when new evidence appears.
So the system can distinguish between:

Why did this stage fail?

and:

What did we learn that should change what happens next?

That second question can shorten the route.
It can also make the route stronger.

If implementation discovers unexpected UI impact, I may want to add a UI/UX review that was not previously relevant.

If the design has gone stale, I may want to go all the way back to design and approval.

If a dependency was updated after a security finding, I may want security and affected validation again, but not product-intent review.

The goal is not “skip more stages.”

The goal is:

Run the stages that still add evidence.

Sometimes the correct route is no route at all

There is another case I run into more often than I expected: the issue should not continue at all!

An issue can become stale.
Another change may already have solved it.
The original requirement may no longer fit the product.
Or it may turn out to be outside the scope of what this workflow should do.

That is why IssueFlow has NOT_NEEDED as a first-class outcome in addition to PASSED and FAILED.

A NOT_NEEDED result can carry reasons such as:

superseded
obsolete
already done
out of scope

That matters because “failed” means the intended work still needs to happen but something stopped us.

“Not needed” means continuing the work may itself be the mistake.

Sending a superseded issue back through implementation is not recovery.
It is just a very sophisticated way to waste money.

This is mostly an efficiency problem for me

I am building all of this because the economics started bothering me.

The code itself can be produced very fast.

Then I watch the rest of the machine spend time validating, reviewing, retrying, waiting, rebuilding context, and occasionally asking me questions where my contribution is approximately zilch.

At one point my rough mental split felt close to 50/50: creative implementation effort on one side, validation/recovery/administration on the other.

That is not a benchmark. I am now measuring it properly.

But it was enough to make me ask whether the workflow was doing too much work simply because that was the path it had always taken.

And yes, there is a token bill attached to that.

Some days I look at the wasted retries and think I could probably buy another computer if I could just get back the money spent proving the first computer did not need to do anything.

Human attention is part of the routing problem too

I like building software.

I do not particularly enjoy babysitting a merge job because GitHub decided this was a good moment to have a bad day.

And, given the choice, I would rather spend that time with my wife.

That does not mean I want humans removed from the process.

Quite the opposite.

I want the developer involved when the developer has something valuable to add.

Bring me in when:

the design no longer makes sense
product intent is ambiguous
a security trade-off needs judgment
the UI behavior is questionable
a risk decision needs an owner
or the issue may no longer be worth doing.

Please leave me alone when:

the runner needs a retry
GitHub is rate limiting us
a known recovery pattern has already been codified
or the system already has enough evidence to choose the next safe step.

The ideal path is not zero human involvement.

It is human involvement exactly where judgment changes the outcome.

What I want to measure next

The dynamic-routing mechanism is shipped.
The efficiency improvement is still a hypothesis.

I want to measure whether it actually reduces:

repeated stages after narrow rework;
total agent tokens per accepted change;
elapsed time from implementation to accepted result;
retries by failure category;
validation/administration share of total spend;
and low-value developer interruptions.

And I need to watch the other side too.

If the workflow gets cheaper because I skipped useful evidence, that is not optimization.

That is just a cheaper bug.

The quality bar stays.

The question is whether the path to that quality bar can get smarter as the workflow learns.

The question I keep coming back to

AI has made code generation fast enough that the expensive part is easier to see.

It is not always writing the code.

It is deciding what to do with the evidence that arrives afterward.

Retry?
Rework?
Redesign?
Repeat validation?
Add a review?
Skip a review whose evidence is still valid?
Stop because the issue is no longer needed?
Or bring in a developer because this is finally the point where human judgment matters?

Now I have a new level of freedom and need to experiment and optimize, so the question i am asking myself is:
When one of the agent stages fails, which parts of the remaining path still add evidence?

An agent can suggest a lesson. Should it become a rule?

Motty Chen — Tue, 13 Oct 2026 16:53:40 +0000

If you are running an agentic workflow for coding, I am pretty sure you already know that - sometimes an AI coding agent can get stuck for a stupid, boring reason.

For example, in a recent run of my own workflow, an agent failed because it was missing Jam credentials. I use Jam to report bugs and provide a link and the MCP to the triage agent to help it understand the bug, but this run failed even though there was no Jam link to investigate. If anyone with a common sense would have worked on this issue (i.e. a human) they wouldn't worry about it, but the agent does unless you tell it: Before worrying about access, you needed to check if that access is even relevant.

So in this specific instance the agent stopped, raised a huma-gate, and waited for direction. Once I responded, I would expect the agent to remember that decision... That is the sort of correction I want the next agent to use. But turning every correction into a permanent instruction creates another problem: a growing pile of rules, with nobody checking whether they still belong.

Since I am the one building IssueFlow, this was a practical workflow problem for me. Here is how I approach it.

Start with a specific mistake

“Be more careful” is not much of a lesson. It gives the next agent no useful test.

“Do not block on missing Jam credentials when the issue has no Jam link” is more concrete. It names the condition and the action to avoid.

The important question is whether the proposal captures something useful beyond one task. A workaround for a single issue should not quietly become a rule for every future run.

A proposal is evidence, not an instruction yet

The solution I was after is a balance between being too specific and too broad. I didn't want to send an agent a pile of 200 rules with every prompt. Those will get lost in the sea of instructions, not to mention token cost.

In IssueFlow, agents can submit learning proposals. A librarian agent collects, combines, and generalizes them into readable candidate rules, with the actual agent instructions attached.

The IssueFlow operation (a human, lol) reviews both parts: what the rule means, and what the agent will be told to do. I can approve it, edit it or reject it.

That was an important part I did not want to skip. An agent can draw the wrong lesson from a failure just as easily as it can draw the wrong conclusion while coding. “This failed once” does not automatically mean “never do this.”

A useful rule should explain the mistake it addresses, the condition where it applies, and the behavior it asks for. It should leave enough evidence to check the reasoning.

Choose where the rule belongs

The next part in the learning system design was: which agent should even get those rules. For example, a lesson from 'design-review' does not necessarily belong in 'cleanup'. A rule about implementation does not need to take space in every unrelated stage.
Proposals are being received from a specific agent at a specific stage, and when the librarian works on those learning (generalize, consolidate), it does it for each agent separately.

IssueFlow review process lets me choose specific stages or make a rule general. The stage labels make that scope visible, and I can adjust it.

This is a small decision with a real purpose: keep an instruction close to the work where it matters. More prompt text is not automatically more guidance.

Keep reviewing the rules

Approval is not the end of the process. I wanted to measure the effectiveness of each rule.

Agents report when they use a rule. The librarian can use that history to identify rules not reported as used for a long time, and then suggest retiring the rule. I can also retire a rule manually.

Not only does rule approval matter. Rejection has a purpose too. When I reject a proposal with a reason, that reason is recorded and used by the librarian when it handles similar proposals later. Otherwise, I could spend a whole week just rejecting the same idea in a slightly different hat.

Note: the usage reports are signals to investigate. They are not treated as proof that the agent followed every instruction, or that a particular mistake would certainly have happened without it.

Learning needs human judgment too

What I learned after implementing the self-learning system is that this provides a way to maintain instructions. It is not a promise that agents stop making mistakes, but it does help eliminate many of the recurring problems from real-life experience.

The goal is: make a useful correction available to later work, keep its scope clear, and give people a way to remove advice that no longer helps. Major learning finds its way to the prompt and is retired as a rule.

That leaves a question I’m still interested in: how much instruction should survive from one task to the next?

If you use coding agents, how do you handle that today? Do you keep adding instructions, or do you also review and remove them?

Watch the short IssueFlow workflow demo. It shows a design approval and an unexpected decision during a real run. (The learning process described here is a separate part of the product)

My AI marketing agent reviewed my coding agent

Motty Chen — Wed, 07 Oct 2026 22:30:12 +0000

I asked an AI agent to help me market IssueFlow. It ended up reviewing designs from my coding agents—and catching problems in how we planned to measure signups, useful activity, and payments.

That wasn’t quite the marketing help I had pictured. But here we are.

I use ChatGPT to help manage the campaign: find the right people, explain the product, and help bring in customers. Ideally while I’m doing my full-time job and building two products.

Looking at my calendar, then at my wife, the obvious conclusion was: “I could use some help.”

So we started running ads.

Then my marketing agent pointed out something inconvenient: clicks are nice, but what happens afterward?

Did someone sign up? Did they complete a useful workflow? Did they become a paying customer?

Fair questions. Answering them properly needed changes inside IssueFlow.

So I asked the marketing agent to create the development issues. IssueFlow—which I also use to develop IssueFlow—picked them up. Its configured agents started working through triage, design, and design review.

Then the first design reached an approval gate.

That’s when things got interesting.

The requester became the reviewer

Normally, I review the work at that gate.

This time, the marketing agent had written the issues. It knew what it needed and why. So I explicitly authorized it to review the designs and approve them—or send them back.

I delegated that decision for these specific issues. The system didn’t quietly decide that human approval was an outdated concept.

Now we had one agent requesting a feature, other agents designing it, and the requesting agent checking whether the proposed solution actually solved its problem.

An agent needed an agent.

We had recreated a small software team. Thankfully, nobody scheduled a recurring alignment meeting.

A passing design review wasn’t enough

The first issue covered signup tracking.

Its design had already passed an automated review. But when the requesting agent checked it against the original requirements, it found gaps.

How would the website pass the visitor’s identity and session information to the portal? What should happen if an event was sent, but delivery couldn’t be confirmed? How would later activity stay connected to the original visit?

One subtle problem concerned time.

The design used the time when information was handed over. The reviewer needed the timing tied to the actual visitor session.

Those sound similar until someone leaves a tab open, returns later, and your neat little assumption starts doing interpretive dance.

The agent sent the design back with specific corrections.

An honest confession: I wouldn’t have caught all of this myself. I know how IssueFlow handles, well… issue flow. The details of connecting it to marketing platforms are a different area.

The revised design fixed several problems. The marketing agent accepted those changes, kept the remaining concern open, and requested another focused revision. IssueFlow routed the work through its design-rework stage.

Eventually, the design reached a point the requester could approve.

What mattered was that the feedback stayed with the work. The next review could focus on what had changed instead of starting the whole explanation again.

The most useful questions were about what the numbers meant

The next two issues covered activation and payments.

For activation, one proposed interpretation could count breaking a request into smaller child issues as meaningful progress.

But that wasn’t what we wanted to measure.

Creating more work is not the same as completing useful work.

If it were, my to-do list would be our most successful customer.

The requesting agent pushed back: activation should reflect a successful workflow outcome. Splitting a request into pieces wasn’t enough.

At this point, I was watching agents discuss what “useful” actually meant. A surprisingly familiar software-development conversation.

The payment design raised another question.

Suppose you can verify a payment, but you haven’t finished checking the earlier payment history. Can you call it the customer’s first payment?

You know money arrived. You don’t necessarily know it was the first time.

That distinction matters when you’re counting new paying customers.

The revised design kept those facts separate instead of turning uncertainty into a confident-looking number.

There were also corrections around consent and recovery. Analytics needed to respect consent, and a measurement failure shouldn’t prevent the customer’s actual workflow from succeeding.

These weren’t just code-style preferences. They were decisions about whether the implementation would still mean what we originally asked for.

Approval still wasn’t a magic wand

Once the designs were approved, implementation continued.

And later checks found more problems.

Code review caught implementation defects. Validation exposed missing test evidence. Related changes needed to be brought together so they would work correctly.

IssueFlow routed work back for corrections as those checks required.

A design gate doesn’t make everything afterward correct. It gives the work a clearer direction. Later checks still have a job to do.

All three issues eventually completed, and their code reached a successful production deployment.

Even then, we couldn’t announce, “Measurement solved!”

Configuration and verification that the events were actually received and processed remained separate work.

Shipped code and a verified customer journey are different milestones.

That’s a distinction I want both the system and the people using it to keep making.

What I’d keep—and what I’d improve

I build IssueFlow, so this isn’t an independent customer review. It’s an account of using the product to develop the product.

And it wasn’t frictionless.

Some reviews required too much reading. A short summary of “what you asked us to fix, what changed, and where the evidence is” would have made the decision easier.

Still, three things were useful:

The requester checked the meaning of the work. A technically reasonable design could still measure the wrong thing.
Feedback stayed with the issue. Revisions could address specific concerns without losing the earlier decisions.
Approval had a clear scope. I chose which decisions to delegate. That didn’t remove the need for later checks.

I didn’t personally type every review response. I decided who could make those decisions for this work.

That’s the control I care about: being able to delegate execution and judgment deliberately, with clear boundaries and a record I can inspect.

Also, apparently, having an AI agent discover the timeless software-development experience of saying:

That’s close. But it isn’t quite what I asked for.

Some things really do survive every technology shift.

Want to see the workflow?

This short demo follows a separate IssueFlow issue. It shows a planned design approval, an unexpected agent question, and the recorded decision that lets work continue.

Watch our short demo.

If you’re building with coding agents, who checks that “done” still means what you originally wanted?

I’m the developer behind IssueFlow, a workflow orchestration tool for AI coding agents. It helps automate the development process while keeping human judgment where you want it. This story comes from using it to build IssueFlow itself.

AI Made Me Faster at Building—But It Also Gave Me More to Supervise

Motty Chen — Fri, 25 Sep 2026 15:34:06 +0000

I did not set out to build Issue Flow as a product.
I built it because I was frustrated...

In September 2025, I was developing a large product while working a demanding full-time job. This is a product I was planning to build for a long tine, but never had the bandwidth. AI had opened up a new kind of possibility: I could make meaningful progress in the limited time I had outside work. With an IDE, a chat interface, and early coding agents, I started building piece by piece.

But as development accelerated, a different problem appeared.
The agents could help write code. They could help investigate. They could help with individual tasks.
What they could not do was run my entire development process in a way I trusted. With over 35 years in IT, I have my quality requirements I did not want to compromise on.
I tried the tools that were emerging at the time: Copilot agents, Cursor, the Kiro, and others. Each solved part of the problem. But none gave me what I actually needed: a way to orchestrate work from ideation through deployment, while involving me only when my judgment was truly needed. Later, when the product was testable, the problem increased - bug reports came with very vague language, so I had to replicate them one by one to recreate the issue myself.

I did not want to spend my limited time babysitting prompts.
I wanted to decide where the human checkpoints belonged. I wanted agents to keep moving through clear work and call my attention when there was a decision, a blocker, a quality concern, or a meaningful tradeoff.
That was the beginning of Issue Flow.

The real problem was not code generation

When people talk about AI-assisted development, the conversation often centers on how quickly an agent can generate code.
That matters. But once you are working on a real product, code generation is only one small part of the process.

Someone still has to understand an issue. Someone has to determine which repository owns it. Someone has to investigate the root cause, consider dependencies, sequence work, validate the result, and decide whether it is ready for production.

New features came with their own challenges: intent alignment, business value, multi-repo coordination, understanding the persona, and other challenges the AI could not do on its own - not at the quality I needed.

For a builder with limited time, this coordination becomes the bottleneck.
At first, I created something simple for my own workflow: a way to orchestrate agents around the steps I was already taking. I designed it to stay flexible because I knew my process would evolve as the product evolved.
And it did.

The early versions had plenty of problems. Agents could crash. Detecting a completed task was not always reliable. Capacity management was inconsistent. Each failure exposed another part of the workflow that needed to become more resilient.
By May 2026, I realized I needed to take it seriously as a product in its own right.

I gave it a name: Issue Flow.

From vague report to production

The need became even clearer when my original product, Aligned on Purpose, reached the stage where testers could use it.
Testers reported issues from their own perspective, as they should. But a report such as “this screen is not working correctly” does not tell a developer whether the problem lives. Is it in the frontend, the backend, an API contract, or a different repository altogether?
I found myself doing so much manual research that it sometimes felt faster to test the whole product myself.
So I expanded Issue Flow.

It could take a vague issue, gather available evidence such as screenshots, network activity, and logs, determine the likely root cause and responsible repository, and create the appropriate sub-issues. It could account for dependencies and concurrency before moving work into the system.

Over time, it became a workflow that could take a bug report or enhancement from the initial description toward production, while bringing me in only where I could contribute real value.
That distinction matters.

The goal is not to remove humans from development. The goal is to make human involvement intentional.

Control without constant supervision

As Issue Flow became more capable, I ran into another challenge: cost.
More automation meant more work could move at once. More work meant more token usage. Eventually, a quota limit could stop a workflow in the middle of its progress.

So I added another layer of orchestration: choosing models based on issue complexity, recognizing quota limits, and switching to another available agentic framework when appropriate.
Issue Flow is not meant to lock people into one AI provider or one way of working.

Whether someone prefers Gemini, Copilot, Claude, Cursor, OpenCode, or another framework, they should be able to design a workflow that matches how they work. They should be able to define their own quality gates, approval points, escalation paths, and budget controls—without maintaining a maze of configuration files.

Building the product with the product

Today, I use Issue Flow to build Aligned on Purpose.
I also use Issue Flow to build Issue Flow itself.
That is the most meaningful validation I could ask for. What started as a personal solution for building alongside a full-time job has become a product for other developers and teams facing the same reality.

AI can make us dramatically faster.
But speed alone is not the answer.
The real opportunity is to build workflows where quality stays protected, budgets stay visible, and people are involved exactly where their experience and judgment matter most.
That is why I built Issue Flow.