Disclosure: I’m Soham, founder of Builderr. We’re running Signalpost, a challenge for agents that research Norwegian companies. I want to share the design question behind it, because it’s easy to miss when a result looks polished.
A company-research agent can return a real fact, cite a real page, and still be wrong if that page belongs to a parent company or a similarly named business. A useful profile needs to get the entity right before it gathers facts. It also needs to show where each fact came from, when it was checked, and what it could not verify.
In Signalpost, the agent receives a Norwegian company number and must return a complete profile from public information. The challenge scores coverage, fact accuracy and evidence, synthesis, and ease of checking. A material wrong-company match blocks qualification. Every entry is tested against the same checked reference collection, so the comparison is about the submitted agent, not who picked the easiest companies.
The hard parts are practical: match the exact entity, keep the source and retrieval date with every claim, preserve useful history when a profile is refreshed, and return a clear result even when information is missing. I’m especially interested in failure cases that could make a well-written profile misleading.
If you build research or browsing agents, what would you test first: parent/subsidiary matching, stale facts, coverage of smaller companies, or something else?
The challenge is open until October 21, with $2,500 in final rewards. The brief, scoring rules, and starter are here: https://builderr.ai/challenges/signalpost
Keep building,
Soham
Top comments (0)