DEV Community

US Directory
US Directory

Posted on

Designing a scoring method you can audit: notes from US Directory

US Directory describes local service businesses using public records, and it publishes the method behind every listing. A few design choices make that method something a reader, or a developer, can audit instead of trust blindly.

Missing data is not a zero

When a fact cannot be checked, the factor stays unscored and its weight is redistributed to the factors that could be checked. A company is neither punished nor rewarded for a gap in the record. Positive-practice factors with nothing to read get a neutral value capped at 0.5, so silence never looks like excellence.

Evidence has a strength

Every fact carries the strength of its origin: 1.0 for an agency record, 0.8 for a file published on the company's site, 0.6 for a precise statement, 0.35 for a loose claim. Values that rest on site statements alone are capped at 0.6.

Additive contributions

The score is a weighted average over checked factors, and each factor's contribution is reported separately. Because contributions sum exactly to the score, a reader can see which factor moved it and by how much.

Score and confidence are separate numbers

Confidence measures how much of the picture was checked and how strong that evidence is, with a bonus for independent source families. A weak license match lowers confidence without changing the score.

Ranks must survive noise

Before a list shows rank numbers, the weights are randomly redrawn 500 times. If the leading places shift too much, the list falls back to tiers without numbers.

Versioned and machine-readable

Factors have stable identifiers; a factor whose meaning changes gets a new one. The full method is published as a machine-readable file in a public repository, alongside the human-readable text.

Read the methodology at usd.org.

US Directory

Top comments (0)