I run a small site that aggregates something boring: Roblox game codes. The only
thing that makes it different from the dozen larger sites doing the same job is a
rule I wrote for myself.
A code only goes on my page if at least two independent sources list it as
working, and no majority calls it dead.
Simple enough to implement in an afternoon. The word that turned out to be hard
was independent.
The day the rule fired for the wrong reason
Two codes showed up for one game. Two sites listed them. Two is my threshold, so
in they go — except I looked at the two sites first.
One was Pocket Tactics. The other was PCGamesN. Both are Network N properties.
Same company, same code desk, in practice the same list published twice.
That's not two sources. That's one source with two domains, and my rule would
have been satisfied by a single editorial decision at a single publisher. The
whole point of requiring two was to make a single mistake insufficient.
So I didn't add the codes, and I wrote down why.
The harder case: two sites with no visible relationship
Shared ownership is the easy version — you can look it up. The version that
actually worries me is syndication: site A licenses or scrapes site B's table,
and nothing on either page says so.
Here's the test I settled on, and it needs no external data:
Compare the two tables as sequences, not as sets. If the entries appear in
the same order and the reward wording matches word for word — including in
the expired section — treat them as one source.
Order is the load-bearing part. Two teams independently tracking the same game
will arrive at overlapping sets; that's expected and it's exactly the agreement
I'm trying to measure. What they will not do is independently arrive at the same
arrangement, and they will not independently phrase a reward as
"5 Exotic Ingots, 10 Reforge Stones" down to the comma.
The expired section is the best signal of all. Nobody curates their dead-code
list. It's append-only junk, ordered by whenever someone got around to moving a
row. When two sites' junk drawers match, they're the same drawer.
On one game I found two sites whose active sections agreed on 28 of 33 entries in
identical order, with identical reward wording — and whose expired sections had
all nine rows in the same sequence. I stopped counting them as two.
The part I got wrong, and had to write into the rule
I assumed "these two sites are the same source" was a property of the sites.
It isn't. It's a property of the site pair on a specific topic.
I had flagged one pair as syndicated based on their table for game A. Later I
checked the same pair on game B, expecting to confirm it. They had 12 entries
versus 17, different sets, different reward wording line by line. On game B they
were plainly independent.
Which makes sense once you say it out loud: a site can republish someone else's
table for the games it doesn't cover itself, and write its own for the ones it
does. So the rule now reads "this pair, on this topic, as of this date," and I
re-check rather than carry the verdict forward.
If you're building anything that scores agreement across sources, I'd suggest
assuming this is true of your sources too until you've checked per topic.
What the rule looks like when it refuses to fire
Three days, same rule, three different outcomes. I think the third is the one
worth talking about.
Day one — it rejected. Two listings, one publisher, codes held back.
Day two — it accepted. Two more sites, genuinely unrelated, published the
same codes. Threshold met, codes added. Same rule, opposite direction, one day
apart. That mattered to me, because a rule that only ever says no is
indistinguishable from being slow.
Day three — it declined to act. A code I'd been carrying got moved to the
expired table by two independent sites. Under the symmetric version of my rule
that's enough to flip it. But this particular row wasn't something I had taken on
anyone's word — I'd typed it into the game myself, on my own account, and it was
accepted. I wrote that down at the time, with the date.
So I left the row alone and published the disagreement underneath it instead:
who says dead, who says alive, who silently deleted the row, and what I actually
did and when.
The general form, which I think applies well outside my little corner:
Aggregated evidence updates aggregated claims. It does not overwrite
first-hand records. Those are different tiers, and collapsing them means a
loud enough consensus can erase something you observed.
I'll be honest about the limit: my own observation was ten days old, and I have
not re-checked it. The row is not a claim that the code works today. It's a claim
about what I saw on a specific date — which is the only kind of claim I can
actually support.
Practical leftovers
If you're weighing sources programmatically:
- Define "independent" before you count to two. The number is the easy part.
- Check ownership first — it's a lookup, and it catches the cheapest failure.
- Compare sequences, not sets. Sets converge honestly; orderings don't.
- Look at the junk sections. Nobody curates those, so they fingerprint well.
- Scope the verdict to the topic, and re-derive it rather than caching it.
- Keep your own observations in a separate tier from what you aggregated, and never let the aggregate silently overwrite them.
- Publish the disagreement. "Three sources say X, one says Y, here are the dates" is more useful to a reader than a confident single answer, and it's honest about what you actually know.
The site is codereceipts.com if you want to see what
this looks like rendered. Every row carries where it came from and when.
I'd genuinely like to hear how other people handle the syndication problem —
the sequence test is cheap and it works, but it's a heuristic, and I'm sure it
has false positives I haven't hit yet.
Top comments (0)