Paste 104.28.28.5 into our IP checker and you get 65 out of 100, amber, with a relay tag sitting next to vpn and proxy. Paste a Mullvad WireGuard exit in Atlanta and you get the same score. One is an Apple egress carrying ordinary Safari traffic from people who flipped a switch in their iCloud settings. The other is a node on a VPN people pay for.
It looks like a bug. We looked at it as one, and then decided to keep it.
What the vendors actually return
Our score is a weighted average across a set of reputation sources, plus hard floors. The floor that matters here: if enough dedicated sources vote proxy or VPN, the score can't drop below 65. "Dedicated" means everyone except ip-api, whose flags are coarse booleans. For an address that already sits in a datacenter, "enough" means two votes, because a lone proxy flag on cloud space is usually one vendor painting a whole range.
On the Apple address, three sources voted: Scamalytics, proxycheck and vpnapi.io. The odd part is what two of them said next to the vote. Scamalytics returned is_apple_icloud_private_relay: true and flagged the IP as a VPN in the same response. vpnapi.io returned relay: true and vpn: true together. So they know exactly what this address is, and they still call it a VPN.
You can check it yourself, no key needed:
curl -s 'https://ipok.io/api/ip?ip=104.28.28.5' \
| jq '{risk, signals, evidence, floors: .riskBreakdown.floors}'
The weighted average of the individual source scores comes out around 30. The floor is what drags it up to 65.
What we tried
The first fix was the obvious one. We already detect relay: Apple publishes its egress ranges and we load them into our offline list, and IPHub and vpnapi.io both expose a relay field. So exempt relay IPs from the VPN floor and move on.
We shipped half of that. Relay now exempts an IP from the hosting floor of 35, which exists because datacenter traffic gets treated with more suspicion even when nothing specific is wrong with the address. That floor was plainly wrong for Private Relay. The exit is in Cloudflare's address space, but there's a normal person behind every connection. Our own offline list also stopped casting a VPN vote on those ranges.
Dropping the 65 was a different question. The score isn't a verdict on whether the person behind the IP is up to something. It's a forecast of how other services will treat the address. Any site that asks one of those same vendors gets "VPN" back, relay or not. If we removed the floor, we'd be overruling three vendors whose verdicts other sites may well be reading. The score would look friendlier and predict worse.
What we do now
The decision sits in the code, right next to the check, so nobody "fixes" it later:
const isRelay = any("relay");
// Relay only exempts the hosting floor, NOT the 65 proxyVpn floor.
// The score describes how the outside world treats this IP,
// and many services really do block iCloud Private Relay.
if (any("hosting") && !hasTrustedWhitelist && !isRelay)
floors.push({ key: "hosting", floor: 35, bucket: "contextual" });
(The comment is translated; the original is in Chinese.)
The relay tag in the result is the explanation, and the methodology page has a paragraph saying the same thing in plain language.
This has a cost. The headline number can't tell "you turned on an Apple privacy feature" apart from "you're on a paid VPN". If that difference matters to you, you have to look for relay in signals, or open the per-source raw fields and read Scamalytics' relay boolean yourself. We accepted that. The alternative is a score that disagrees with the vendors doing the actual blocking, which would make it wrong about the one thing it is supposed to predict.
If you run any kind of reputation scoring, it's worth deciding which of the two your number is: a judgment about the user, or a prediction of how others will judge them. We went with the prediction, and Private Relay is where that choice feels worst to explain to a user.
I built ipok, the checker used in the examples above.
Top comments (0)