DEV Community

chovy
chovy

Posted on Originally published at dev.profullstack.com

Letting the agent say it does not know

Letting the agent say it does not know

The niche operator thing I wrote about a few hours ago had a hole in it. Someone claims a niche on nichedb.dev, they earn a share of what it makes, and their share goes up as they contribute. Fine. But every way of contributing required them to go and find something to do. Open the dashboard, stare at it, think of something useful. That is a job, and not a very appealing one.

So the agent asks instead.

It is building or researching for a niche, and it hits something that only a person who has done the job can settle. Steam lists an early access date and a 1.0 date, and only one of those is the release. Two roofing sources disagree about waste percentages. It cannot read its way out of that. It asks the human running the niche, they answer in about two minutes, and the answer becomes both niche knowledge and a scored contribution. The agent's ignorance is what generates the operator's income, which is a nicer shape than the usual one.

The part I spent longest on is what happens when they cannot answer.

There are three buttons: answer it, say there is not enough context, or send it back and ask the agent to research more. The last two are worth zero points. That is deliberate and it took me a while to be sure about. If answering is the only thing that pays, then somebody who half knows will guess. A guess that gets verified becomes wrong data in a set that other people pay to read, and nothing downstream can tell it apart from the real thing. An unanswered question costs a day. A confident wrong answer costs the niche its credibility. So declining is free, and the page says so underneath the buttons.

Two bugs, both mine, both found by running the thing rather than testing it.

The first was in code I shipped this morning. Reversing a fraudulent contribution marked the row as reversed, which took it out of the score, and also wrote a compensating negative row. Both. So taking back a bogus 100 point entry subtracted 100 from a total that had already stopped counting it, and somebody's legitimate 23 points collapsed to zero. Honest work went down with the fraud. Each half was correct on its own, which is why no unit test had anything to say about it.

The second one was live on the site for about three hours. Bun's Postgres driver passes a string parameter cast to jsonb straight through instead of parsing it, so a column written that way stores the text {} rather than an actual empty object. Every opportunity page was then rendering a section called "How that score is made" containing two dimensions, named 0 and 1, holding a brace each. Object.keys of a string gives you its character indices. I had a note to myself about this exact trap, written weeks ago, and I did not read it until after I had shipped the bug it describes.

One smaller thing worth writing down. Profile pages are at /@handle. Hono does not match a route parameter that sits behind a literal prefix inside a path segment, so /@:handle never fires at all. No error, no warning, the request just falls through to the catch all with the @ still attached. I found it by curling the URL instead of assuming the route existed, which is the only reason I found it.

The loop is live. The endpoint is signed in both directions and answers 503 rather than accepting anything unsigned, because an open endpoint there is a way to mint contributions that pay.

nichedb.dev/opportunities

This post was drafted with AI assistance.

Top comments (0)