DEV Community

Cover image for Why your OSINT tool lies to you
WinstonRedGuard
WinstonRedGuard

Posted on

Why your OSINT tool lies to you

Open almost any OSINT tool, run a username, and you get a wall of green checkmarks. Found on 40 sites. Phone traced to a carrier. Email confirmed. Every line rendered in the same confident styling as a real breach hit pulled from a cryptographic database.

Most of those checkmarks are lying to you. Not on purpose. The tool simply has no way to show the difference between "a cryptographic check confirmed this" and "a web page returned HTTP 200, so I guessed."

The uncertainty is real, and good analysts carry it in their heads. They know a phone "carrier" field is often wrong, that a username hit on an obscure site is close to a coin flip, that an email "exists" only means the domain accepts mail. But that knowledge lives in the analyst, not in the tool. It evaporates the second a junior reads the report, or the result gets pasted into a slide where the green is all anyone sees.

So I wrote a small library that moves the uncertainty out of the analyst's head and into the result itself. The idea is one sentence: every lookup gets wrapped in an envelope, and the verdict is capped at the highest level the source type can honestly support. Not the highest level this particular result reached. The highest that kind of source can ever reach. And the cap lives in the code, so a downstream UI cannot accidentally promote a guess into a fact.

There are four levels: verified, inferred, heuristic, unverified. The ladder is the easy part. The useful part is where each source type is forbidden from climbing.

A phone number can never return verified. Not with libphonenumber, not with a messenger-presence hit, not with a paid reverse-lookup. Number portability already broke the prefix-to-carrier guess years ago. A WhatsApp hit proves the number is reachable, not who owns it. A paid lookup gives you a current carrier, never an identity. So the wrapper caps at inferred and ships a warning you cannot switch off: number_portability_not_reflected.

An email can never return verified either. An MX record proves the domain accepts mail. It does not prove that this mailbox exists, that a human reads it, or that your target owns it. Aliases, forwarders and catch-all rules are invisible from the outside. SPF and DMARC describe how the domain wants mail handled, not who owns the address. Cap: inferred, with a permanent mailbox_existence_not_proven.

A username hit from 404-scraping is heuristic by construction, because status-code detection is fragile and false positives are the normal case. It only earns a promotion when independent platforms agree, or a per-site reliability history backs it up. A single hit on one site stays a guess, and the tool says so.

The same logic runs the other way for the sources that deserve trust. A DNSSEC-validated domain resolve, a TLS handshake, a HIBP k-anonymity password check: those are real, and they reach verified. The point was never to be cynical about everything. It was to stop one honest signal and one lucky guess from rendering identically.

And the pipeline that combines all of them inherits the verdict of its weakest link, because an investigation is only as trustworthy as its shakiest input.

This is the part most tools skip. The library does not claim its own numbers are accurate. The confidence anchors are hand-calibrated tradecraft, not measured precision and recall. They have not been validated against a labelled corpus, and the README says exactly that, in plain text. They express a relative ordering, an MX record beats a regex match and a DNSSEC resolve beats both, not a probability you should bet money on. A tool whose entire job is fighting overclaiming has no business overclaiming about itself.

None of this is exotic. It is the stuff every OSINT analyst already knows and every OSINT interface throws away the moment it renders a result. The only real move is refusing to throw it away: put the ceiling in the code, ship the caveat as a warning nobody can silence, and stay honest that the numbers are judgment, not measurement.

If you build OSINT tooling, surface the verdict and the warnings. Not a bare green checkmark.

osint-trust-envelope is a small, zero-dependency Python library. It performs no lookups of its own; you bring the raw result, it assigns the honest ceiling. https://github.com/WRG-11/osint-trust-envelope

Top comments (0)