I'm building a research tool for "should we use X or Y" decisions. It browses the docs and pricing pages, cites the source on every line, and then a second pass checks each claim against the page it cites.
That second pass exists because of my own early demo: a clean, fully cited comparison table where 4 of 7 claims were contradicted by the pages they cited. Since then, the checks keep catching things like:
- Prices hidden behind controls. A worker called a price "not stated" because it sat behind a volume dropdown, and it missed a free plan's one-user cap that was only in a comparison-table column. Reading pricing pages properly turned out to be its own skill.
- Content rendered by script. Asked which XMPP clients support a messaging spec, two workers said "unsupported" for clients whose own machine-readable feature files say "complete", because the support table on the page only renders behind a "Show" button.
- Right number, wrong meaning. A claim that a client "implemented version 8.0.0" of a spec: 8.0.0 was the client's release that added it, not the spec version. The checker now grades a number used with the wrong meaning as contradicted.
What I'd like: 10 real decisions to run it on. Comment with one you're facing, plus constraints (budget, volume, stack), and I'll reply with the cited answer within a day, gaps marked. Free. The useful part for me is hearing where it's wrong.
Top comments (0)