DEV Community: Trakshan Mishra

My AI agent's 34 tests passed. Then I broke the code 31 times.

Trakshan Mishra — Thu, 05 Nov 2026 04:41:22 +0000

"All tests passing" is the most common line in an AI agent's final summary. I wanted to know what it is worth.

Disclosure first: this week I'm working with the team behind Lemon Code, a free coding agent. That's exactly why I didn't want to stop at "it passed".

The build

I asked it for a weather and travel planner: search a city, show the 7-day forecast, pick trip dates, rank the best days to go, and suggest a packing list. Plain Vite and vanilla JS, live data from Open-Meteo.

About 3 minutes later I had a working preview. On the way, 3 of its tests failed on the first run. It read the errors, fixed them, and reran. The final summary said 34 tests passing.

Check 1: run the tests outside the agent

I downloaded the zip and ran it on my own laptop.

 ✓ test/storage.test.js (8 tests)
 ✓ test/weather.test.js (11 tests)
 ✓ test/planner.test.js (15 tests)

 Tests  34 passed (34)

Good, but that only proves the tests pass. It says nothing about whether they would fail.

Check 2: delete each function and see who notices

There's a good post here on dev.to, The Test That Survives Its Own Implementation, about agent tests that keep passing after you delete the code they cover. So I tried it.

For each of the 21 exported functions in the logic files, I added return undefined; as the first line and reran the suite.

for name, body_start in exported_functions(file):
    write(file, original[:body_start] + "\n  return undefined;" + original[body_start:])
    failed = run("npx vitest run --reporter=json")["numFailedTests"]
    write(file, original)

Result: 21 out of 21 caught. Every gutted function broke at least one test, from 1 failure (geocodeUrl) up to 8 (parseISODate). No dead weight.

Check 3: small, realistic bugs

Deleting a function is a loud bug. Real bugs are quiet. So I made 10 one-line changes to the trip planner:

Change	Tests that failed
Off-by-one in the day count (drop `+ 1`)	3
Date filter drops the last day (`<=` to `<`)	2
Best days sorted in reverse	2
"End date before start date" check removed	1
Rain penalty halved	0
Wind threshold 20 to 30 km/h	0
Sunny-day bonus removed	0
Snow penalty removed	0
"Too hot" threshold 30 to 35°C	0
Tie-break between equal days reversed	0

4 caught, 6 survived.

The pattern is clear. Dates, ranges and sort order are well tested. The day-scoring rules are not. scoreDay has one test, and it only checks that a clear, calm day scores higher than a storm. Any weight you tweak still passes that.

Also worth saying: the UI code (ui.js and main.js, about 480 lines) has no unit tests at all. The agent checked the UI by driving the app in a browser instead, searching two cities and saving a trip. That's a reasonable choice for a small app, but it's not something npm test will protect.

What I'd take from this

"34 passing" was true, and the tests weren't fake. They caught every deleted function and the date bugs I'd worry about most.
The gaps are where the logic is fuzzy. Scoring rules with magic numbers got one sanity test. That's the kind of code where I'd ask the agent for more: "write a test that fails if the rain penalty changes."
The check is cheap. Breaking 31 things and rerunning took a couple of minutes with a small script. I'd do this on any agent-written suite before trusting it.

If you want to try the same thing, Lemon Code is free and doesn't need an API key: lemoncode.ai. Build something, download it, and break it. I'd like to hear what your tests let through.

PocketTrail: three small things to notice outside, picked by a small open-weight model

Trakshan Mishra — Tue, 06 Oct 2026 14:00:39 +0000

This is a submission for the Hacktoberfest Week 1 "Touch Grass" challenge.

What I Built

PocketTrail is a small web app for people who want to look up from their phone for twenty minutes.

You tell it how much time you have (10, 20 or 30 minutes), where you are going (a park, a garden, your neighborhood, or just a window or balcony), what it is like outside (daytime, evening, rainy) and your mood (calm, curious, creative). You can add a note like "stay seated, no photos". It gives you three short things to notice, with durations that add up to the time you picked.

Then you save the walk to your device, put the phone away, and open it again when you want the next prompt. Saved walks work without a connection. There are no accounts, streaks, maps or feeds.

Demo

Live app: https://pockettrail-6dyu.onrender.com

The hosted demo runs in sample mode. It serves original, hand-written activities and makes no model call, and the page says so. I did not put a paid model or voice key on the public instance. The real model path works locally, which is what the numbers below come from.

Code

trakshan-mishra / pockettrail

A little less screen. A little more outside. Short nature activities with offline saved walks.

PocketTrail

A little less screen. A little more outside.

PocketTrail turns your time, setting, mood and outdoor conditions into three gentle nature activities. Save a walk on your device, then take a pause outdoors or by a window. Saved activity text works without a connection. Optional narration is available when ElevenLabs is configured.

Live demo: https://pockettrail-6dyu.onrender.com (free instance, so the first load after idle can take about a minute). The hosted demo runs in sample mode: it serves authored activities with no model calls and no voice. Model generation and narration need the configuration described below.

Created during the Hacktoberfest Week 1 2026 Touch Grass challenge. This repository covers Week 1 only.

Run locally

Requires Node.js 24 and Python 3.13+.

cp .env.example .env
bash scripts/dev.sh

Open http://127.0.0.1:8000. The script builds the production frontend so offline caching can be tested locally. For interface development run the backend separately and use…

View on GitHub

Run it locally with bash scripts/dev.sh. The README covers Ollama, Backboard and ElevenLabs settings.

How I Built It

Stack. React and TypeScript on the front, FastAPI with Pydantic on the back, SQLite for cache and usage quotas, IndexedDB on the device for saved walks. One Docker image builds the frontend and serves it from FastAPI.

What the model does, and what it does not. My first instinct was to let the model write the activities. I dropped that quickly. A small model gets confident about things it cannot know, and this app tells people to go stand somewhere.

So the instructions are written by me, in a catalog of about fifteen activities per context. The model does two jobs:

Pick three distinct activities that fit the request.
Write a short personal "focus" line for each.

The server then checks the output against the catalog (setting, equipment, evening and rain rules), rejects duplicates and unknown activities, and assigns durations that sum exactly to the requested time. A rainy request can never return a daylight-only activity. A window request only draws from the window catalog. One validation repair is allowed; after that the request fails instead of looping.

Two small models, same cases. I ran qwen2.5-coder:3b and Phi-3-mini-128k through Ollama on my laptop, three identical requests each.

Model	Valid packs	Mean latency
qwen2.5-coder:3b	3 / 3	29.8 s
Phi-3-mini-128k	3 / 3	67.4 s

This is a small pilot, not a benchmark. Both passed the structural checks, so the more useful result was in what the checks missed when I read the outputs. Qwen suggested a stroll in its introduction even though the preferences said stationary. Phi claimed a heartbeat could be synchronized with breathing. I changed the introduction to authored text and added a rejection rule for heartbeat claims. Free-text wording can still get past a keyword check, so a human should read any pack before it goes in a demo. I also found the focus lines from the small model are sometimes generic or repeated. I did not fix that.

Qwen is a coding model. I used it because it was already installed, not because it is the best choice for this task.

Other checks. A script went through all 108 combinations of context, setting, mood and duration against the catalog and schema. A Docker image built and served the health endpoint as a non-root user, and I fixed a volume-permission failure that showed up there.

Keeping spend small. Identical requests reuse a cached response. Every hosted model attempt, including the repair, reserves a quota slot in SQLite before the call is made. Narration only runs when you press "Add voice", and it takes a saved pack ID, not arbitrary text. These are call and character limits, not dollar caps, and the README says so.

Offline. The shell, fonts, saved packs and any downloaded audio are stored locally. Offline phone use covers saved walks. Generating a new pack still needs a connection.

Timers. Walk Mode has an optional timer with a short chime made locally with Web Audio, plus vibration where the browser supports it. I tested this on desktop only. Phones can suspend audio and timers when the screen is locked, so I do not claim pocket reliability.

What I did not finish

A real outdoor trial. I tested the flow in a desktop browser, not on a walk, so I have no field results to report.
A physical phone test of offline reload, vibration and lock-screen behavior.
Hosted model and voice. The code supports Backboard and ElevenLabs, but I made no real calls with either, so I am not listing them as integrations.

Why Does Open Innovation Matter?

The model here is replaceable. It is called through a thin adapter, so the same app ran against two different local models and can point at a hosted one by changing a few environment variables. That made the comparison above possible, and it meant I could read the model card, run it on my own machine, and keep preferences on the device when I chose local inference.

It also made the failures visible. With an open-weight model I could compare two models on the same inputs and see exactly where each one made things up. That is why most of the safety work ended up in the catalog and the validator, and not in a prompt asking the model to behave.

If you choose a hosted model, your preferences go to that provider. With local Ollama they stay on your machine.

stepfix: tech support that never writes the command for you

Trakshan Mishra — Mon, 05 Oct 2026 06:47:28 +0000

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

Most of us have a friend or a parent who calls when the Wi-Fi breaks. Helping over the phone means reading commands out letter by letter, or telling them to paste something they can't read. I wanted something I could send them instead, that is careful in the same way I'd be.

stepfix is a support chat for people who aren't technical. You describe the problem in plain words, like "my Wi-Fi shows no networks". It asks a couple of questions, then walks you through the fix one step at a time.

Each step is a card with:

the exact command to copy
why we're running it
what you should see if it worked
"it worked" and "it didn't" buttons, so the next step depends on what actually happened

You run every command yourself. stepfix never touches your machine.

The one rule I didn't want to bend: the AI never writes a command. Every command comes from a public, reviewed library of 63 scripts covering Wi-Fi and Bluetooth on Ubuntu and Windows 11, plus a few developer-tool fixes. Each one has a risk level, and 39 of them only read information without changing anything. The model picks which step comes next. The app fills in the exact text. If the model tries to slip a command into its reply anyway, a scanner strips it out.

Why so strict? "Open your terminal and paste this" is exactly how a lot of scams work now. Someone who doesn't know what a command does can't tell a fix from an attack. So stepfix only hands out commands that anyone can read on GitHub first.

Demo

The demo is a live run on Ubuntu, unedited:

"My laptop says it's connected to Wi-Fi but nothing loads"
It asks one question at a time (which OS, when it started) and fills in the case panel as I answer
It switches to step-by-step fixing and shows the first card: a read-only check, with "why" and "what you should see"
I click "I ran it" without pasting anything. It won't move on: a check with no output proves nothing, so it asks for the output
I type "it got fixed". It still doesn't mark the case resolved, because nothing was verified
I stop, and get a report I can copy or download. It says plainly that nobody has been contacted

That last part is the point. It would rather stop and hand you an honest report than guess at a fix.

Code

trakshan-mishra / stepfix

AI tech support that never writes the command for you. Open-weight models, a reviewed public script library, you run every step.

stepfix

stepfix is an AI tech-support helper for people who are not technical. It explains each diagnostic or repair step, and the person runs every command themselves.

Safety model

The AI is never allowed to write commands. Every command comes from the reviewed, public script library in library/*.yaml, which is linted for dangerous patterns. This is the opposite of “paste this into your terminal” scams: the model selects a bounded step, while the application supplies the exact command and explains what it does.

Models

Open-weight models only: gpt-oss-20b and gpt-oss-120b on Groq, with GLM-4.7-flash on Cloudflare Workers AI as the fallback, and a scripted playbook if every model is down. No closed models are used.

Run locally

You need Node.js, npm, a Cloudflare account with Workers AI access, and a Groq API key.

npm install
cp .dev.vars.example .dev.vars
npx wrangler login
npm run dev

Fill in the required values…

View on GitHub

How I Built It

stepfix runs on Cloudflare Workers with Durable Objects, using the open-source Cloudflare Agents SDK and the AI SDK. Every support session is its own Durable Object. A separate Coordinator object handles admission and keeps a quota ledger, so a free-tier app doesn't fall over the moment a few people show up.

There are two roles:

Support asks questions and fills in a case file: OS, problem area, symptom, when it started, what changed. It can't hand off until the case file is complete.
Technician only sees the scripts for that OS and problem area, and recommends one at a time based on what the previous step showed.

Models: gpt-oss-20b for support and gpt-oss-120b for the technician, both on Groq. GLM-4.7-flash on Cloudflare Workers AI is the fallback. Before every model call, the router reserves quota. If every model is down, a scripted playbook continues the case instead of leaving the person stuck.

The script library is YAML. A linter rejects dangerous patterns, like piping a download straight into a shell or recursive deletes. It's then compiled and hashed, so the app only serves what's in the reviewed file. There are 462 unit tests, two browser tests that click through a full repair and a stop-and-report, and 13 simulated support conversations.

The best part of this weekend was a bug. When I first switched from mock mode to real models, stepfix answered nothing. No error, no message, just silence.

It turned out the quota ledger used JavaScript Maps, and the Durable Object saves its state as JSON. JSON.stringify(new Map()) is {}. After a restart, the ledger came back as empty plain objects, the first .get() threw, and the error was swallowed before it reached the user.

The fix had three parts:

maps that serialize properly
loading old broken state without crashing
falling back to the playbook instead of going silent

While testing that, I found the technician had never actually seen the script catalog. It was loaded with require(), which doesn't exist in a Worker, and the catch hid that too. Two silent failures in one path. Now there's a test for each.

Then I recorded myself using it and watched it back. In one reply, the model wrote "run: ip -brief address" right in the chat. My scanner missed it for two reasons:

its list of command names didn't include a single command from my own library
it ignored anything near the word "Wi-Fi", which is every Wi-Fi conversation

Now the scanner learns every command from the library itself, and the exact text from that recording is a test case.

I used AI coding tools along the way, mostly Claude for debugging: going through logs with me and watching my screen recordings back to find where the flow broke. The Map bug and the scanner gaps both came out of those sessions.

Why Does Open Innovation Matter?

Trust. The thing that decides what runs on someone's computer should be inspectable. The script library is public. The models are open-weight, so you know exactly what's behind the chat, and so does anyone auditing it.

The fallback only works because the weights are open. stepfix's main and backup providers are different companies serving open-weight models. When Groq rate-limits, the same kind of model takes over on Cloudflare. With a single closed model, "fallback" means "nothing".

It can go local. Because the models are open-weight, the next step is running the whole thing on the person's own machine, for anyone who doesn't want their system output leaving it. That isn't built yet, but nothing in the design blocks it.

Prize Categories

Overall. I didn't use the partner technologies for this one.