DEV Community: Aghassi Sargsyan

The enum value that had never been written

Aghassi Sargsyan — Sun, 23 Aug 2026 08:33:10 +0000

I found 27 workflow branches that were being skipped while every run still finished as COMPLETED.

The condition on those branches could never match. So the step was skipped, the run was marked success, and nothing ever told me. They had been running like that for weeks.

That is the part worth sitting with. Not that there was a bug — there is always a bug. That the system reported success 27 times a day, honestly, while doing nothing.

The question that made it worse

I wrote about this on a forum and someone asked a question I could not answer:

Does it distinguish "green but semantically idle" from "green and actually processed", or is that still something you catch by comparing runs?

I went to check, expecting to say the data is one level down, in the step rows. Each step has its own status, and SKIPPED is one of the values. So the information had to be there.

It was not there. StepExecutionStatus.SKIPPED had existed in the enum since the beginning of the project and had never once been written. Zero occurrences in the codebase.

A skipped step did not create a database row at all. The loop appended a dict to an in-memory results list and continued. The only trace of a skip lived inside a JSON blob on the execution row.

So:

the step view reads rows, which meant a skipped step was invisible in the UI too
"which runs skipped something?" had no answer short of parsing JSON
a run that skipped every branch and a run that did all the work were identical at every level a human or an alert would look

I had built an enum value to describe a thing, and then never recorded the thing.

The fix, and the flaw in the fix

The fix took an afternoon. A skipped step now writes a real row with the reason. The run reports how many steps it processed, skipped and failed, derived from those rows so the numbers cannot drift from what the step view shows.

Then I added a badge: when a finished run processed nothing, say so.

The same reviewer killed it within the hour, and he was right:

A zero can be legitimate. Quiet day, nothing to send, COMPLETED with 0 processed is correct. The alert gets teeth when the count is two-sided: the run says what it processed, the source says what it handed over, and the two have to tie out.

A scheduled workflow with nothing to do processes nothing every quiet day. My badge would have fired on all of them, been muted inside a week, and then not been there on the day it mattered.

That is worse than showing nothing, because it looks like coverage.

The badge came out. The counts stayed, as plain facts, and the judgement went with the badge.

The same shape, one level up

Two days later I built a deploy script that runs the tests locally and refuses to deploy if any are red. It ends by verifying the deploy.

It printed five green lines:

✓ deployed
✓ server on c15b730
✓ all containers running
✓ agent-mesh.org/health healthy

Every one of them true. The feature I had just deployed was dead. The containers had started before I added the environment variable it needed, so the code shipped and did nothing.

I found it by curling the endpoint and reading the value, which the script had not done.

A deploy verifying that it deployed is not the same as verifying the thing works. Health checks are generic. Effect checks are specific, and specific is the part nobody writes.

What three other people added

I posted the 27-branch story and the thread turned into something better than the article I meant to write.

On maintenance. Per-step assertions do not scale past a handful of workflows, because every assert is another thing to maintain. The answer that survives is to move the assert out of the workflow and into the engine. One place, and every workflow gets it without anyone adding a node.

On the zero, from Yair Sabag. History is a partial answer to the legitimate-zero problem. Do not look at one execution; look at the workflow's own baseline. What does Tuesday 9am normally produce, against Saturday? Zero on a normally-busy slot is a signal. Zero on a normally-quiet one is not. It is probabilistic rather than certain, but it turns blind into suspicious, which is usually enough to know where to look.

The limit, and it is mine: a new workflow has no history, which is exactly the period when someone is most likely to have built the condition wrong. My 27 branches were dead from the first run. There was never a healthy baseline to deviate from.

On cascades. Someone described a self-hosted setup where a request to a local model dropped without a loud error. The node completed, passed an empty payload downstream, and every subsequent node executed successfully against nothing.

That is worse than my case. Mine were 27 independently dead branches. That is one silent failure manufacturing more of them, each of which succeeded honestly, because each did do its job on the nothing it was handed.

It is also why recording what a step received matters as much as what it returned. A step that returns nothing is suspicious. A step that received nothing tells you where the rot started, and those are usually different steps.

And the one I have no answer for. A dedup node with a logic edge case swallowed an entire dataset. The node was correct. The code was correct for the cases it was written for. Nothing in the run is wrong except the number of rows.

Output shape validation does not catch that, because [] and a thousand rows have the same shape.

The pattern

Every one of these is the same thing wearing different clothes: a check that cannot report failure.

A run status that only knows whether an exception was thrown. A test that performs actions and never asserts. A health check that confirms a process is listening. A skipped step with nowhere to be recorded.

There is a smaller version of this I hit in the same codebase. I stored cost as an integer number of cents:

cost_cents = int(
    (prompt_tokens * 0.5 / 1000) + (completion_tokens * 1.5 / 1000)
)

A typical run costs about 0.0955 cents. int(0.0955) is 0. Almost every run I had cost under a cent, so almost every run recorded as exactly zero, and my own analytics page reported 2 cents total across about a hundred executions. I believed it for a while. That is not rounding drift. That is the data being destroyed at write time, by a cast.

What is still broken

The two-sided count. My runs say what they processed; nothing says what the source handed over. The HTTP tool already computes how many records a list endpoint returned and throws it away, because tool results are not persisted per step. Until that changes, a run that processed zero looks the same whether the day was quiet or the condition was broken.

The baseline idea is in the backlog and the data for it already exists. Both are worth doing, and they catch different bugs: history finds drift, source counts find born-broken.

I build a hosted agent platform, which is where all of this happened, so treat the whole thing as biased. But the bug was not exotic and neither was the fix. The enum value was right there the whole time. Nobody had ever written it.

I launched to zero signups, then found 5 features nobody could reach

Aghassi Sargsyan — Wed, 22 Jul 2026 12:55:35 +0000

I spent months building an AI agent platform. I launched it on Product Hunt yesterday. Zero signups.

The comments were friendly. Three of the four asked for the same thing — not features, not integrations, not a lower price. They wanted to see what the agents did and what they cost. One put it better than my own landing page ever did: they liked that it wasn't "a black box."

So I went to make the cost dashboard better. Instead I found out my product had been lying to me for months, and the lies had a pattern.

Here's everything, with the code.

1. Every run cost $0.00

The Cost Analytics page reported $0.02 in total across ~100 executions. I'd assumed that meant the platform was cheap to run. It meant the data was being destroyed at write time.

cost_cents = int(
    (llm_response.prompt_tokens * 0.5 / 1000)
    + (llm_response.completion_tokens * 1.5 / 1000)
)

A typical run on my platform is 56 prompt tokens and 45 completion tokens. That's 0.0843 cents. int() makes it 0.

Not some runs. Essentially every run — because almost every LLM call costs less than one cent. The production numbers: 189 agent runs, 2 with a non-zero cost. 99% of my cost data was zeroes, and the two survivors were just big enough to clear a whole cent.

The rates in that formula were correct. I checked them against the providers' pricing pages; the arithmetic is right. The bug is entirely int() on a value that is almost never ≥ 1. A Decimal would have been the textbook fix, but Decimal / float raises TypeError and ~80 call sites do arithmetic on this number, so I widened the column to a float and kept the unit (cents). It's a dashboard estimate, not money — Paddle handles money — so float rounding is irrelevant here.

2. Workflow costs were never recorded at all

Truncation at least loses precision. This one lost everything.

WorkflowExecution.total_cost_cents and total_tokens_used had no write site anywhere in the codebase. Not a broken write — no write. The columns had been NULL since the feature shipped. The workflow runner computed per-step token counts and dropped them on the floor.

I found this by grepping for assignments and getting no hits:

$ grep -rn "\.total_cost_cents = " src/
$ # (nothing)

The API served the field. The schema declared it. The dashboard summed it. It was never once set.

3. The rates were per-provider, not per-model

if llm_response.provider == "openai":   # gpt-4o-mini billed at gpt-4o rates

Same provider, ~30x cheaper model, same price. Nobody noticed because of #1 — every number was already zero.

4. The cost was rendered in exactly zero components

I fixed the data. Then I went to display it and found costCents had been in the API response for months and was rendered nowhere in the frontend. Not on the run page, not anywhere:

$ grep -rn "costCents" src/ --include=*.tsx
$ # (nothing)

The TypeScript types never declared the field either — so the API was returning data that was invisible to the compiler. No error. Just absent.

And the workflow step list rendered this:

{step.tokensUsed && <span>{step.tokensUsed} tokens</span>}

WorkflowStepExecution has no tokensUsed column. The response schema has no such field. That line had never rendered once. It was in the UI, in code review, in the repo, doing nothing.

5. Analytics had no link. Anywhere.

This is the one that stings.

The app has four analytics pages — Dashboard, Executions, Costs, System. They work. They have a nice sub-nav between them.

Nothing in the entire application linked to any of them. Not the main nav (Dashboard, Agents, Workflows, Executions, Marketplace, Teams, Schedules, Billing). Not the user menu. The only way in was typing the URL. And once you got there, there was no way back except the browser button.

The $0.02 that started this whole investigation was on a page no user could click to.

When I fixed it, I added the back-link to the shared AnalyticsLayout component. Then a browser test told me the link wasn't there. AnalyticsLayout is dead code — all four pages only import a sub-component from that file; the layout itself is never rendered. My fix for the unreachable feature went into unreachable code. I only caught it because I drove a real browser instead of trusting the diff.

The bonus round: a public endpoint

While making Analytics reachable, I exposed a System Health tab to every logged-in user. Looking at a screenshot of my own fix, I stopped: should a normal user see system health?

It was worse than the question. /v1/health/detailed had no auth dependency at all, and the reverse proxy sends /v1/* straight to the backend. Anyone on the internet, no account, could poll:

uptime · Postgres latency · Redis latency · Celery worker count
WebSocket → connections: 1, users: 1     ← live user count

A real-time traffic gauge for my business, free to anyone who cared to watch. Public since the day it was written.

Why it survived every test: the test suite's conftest overrides get_current_user. Any test using the shared client fixture authenticates as a fake user — so a missing auth dependency looks identical to a working one. The endpoint would have passed an auth test. The regression test I wrote deliberately bypasses that fixture:

async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as ac:
    resp = await ac.get("/v1/health/detailed")   # no fixture, no override
assert resp.status_code in (401, 403)

The pattern

Five separate bugs, one shape: built, tested, shipped, unreachable.

cost data → written, destroyed on write
workflow cost → schema'd, never written
costCents → returned, never rendered
per-step tokens → rendered, never sent
Analytics → built, never linked
current_step_id → tracked in the DB, unused by the frontend (still true; it's next)

Every one passed CI. 519 backend tests, 276 frontend tests, typecheck clean. Every one had a plausible-looking implementation. None of them worked end to end, and no test could have caught any of them, because each piece did exactly what its unit test said it should. The column stored an int. The endpoint returned JSON. The component rendered a field. The wiring between them was where the product lived, and nothing tested the wiring.

That's not bad luck. Five in one week is a process failure. Mine was that I verified changes and never verified features — I'd check that the code did what I wrote, not that a user could get to it and see a true number at the end.

The thing that actually found these was mundane: driving the real app, as a real user, and looking at the screen. Every single one was invisible from inside the codebase and obvious from the browser.

What I'd tell past me

A test that mocks your auth cannot test your auth. The fixture that makes tests convenient is the fixture that hides the hole.

"The API returns it" is not a feature. Nobody can see your JSON.

Grep for the write, not the read. A field being read in 80 places says nothing about whether anything ever writes it.

If a number looks suspiciously good, it's probably a bug. $0.02 across 100 runs should have made me suspicious months earlier. I filed it under "efficient" instead of "broken."

Click the thing. Not the diff. Not the test. The thing.

And the launch?

Still zero signups. The observability is honest now, the costs are real, and the pages are reachable — and none of that changed the number, because a product being correct was never what was stopping people. That's a different post, and I haven't earned it yet.

But I'd rather have found all this from four polite comments than from the first customer who actually paid.

The product is AgentMesh — no-code AI agents for small businesses. Zero strangers have ever signed up, so please don't take this as a recommendation. Take it as a warning about your own repo.