DEV Community

98IP Proxy
98IP Proxy

Posted on

Stop Logging Raw Proxy IPs: Measure Exit Churn with Short-Lived Tokens

#privacy #webscraping #testing #networking

An exit-churn test needs to recognize repeated proxy exits, but it rarely needs a permanent archive of raw IP addresses. A safer design converts each normalized exit into a short-lived keyed token and performs the measurement on those tokens.

I work with 98IP. This article focuses on the engineering pattern, not on maximizing rotation or bypassing destination controls.

1. Define a narrow event schema 

{
  "case_id": "pilot-eu-search-v2",
  "observed_at": "2026-08-29T09:00:00Z",
  "exit_token": "daily-keyed-token",
  "address_family": "ipv6",
  "requested_market": "DE",
  "observed_market": "DE",
  "session_mode": "sticky-30m",
  "session_token": "non-credential-session-id",
  "connect_ms": 184,
  "ttfb_ms": 612,
  "result_class": "workflow_valid",
  "retry_number": 0
}
Enter fullscreen mode Exit fullscreen mode

Do not put proxy usernames, passwords, cookies, authorization headers, account IDs, or full response bodies in this stream.

2. Normalize before tokenization

IPv6 has multiple textual representations, so parse the address into its canonical binary form before applying a keyed hash. Do the same for IPv4. Include the pilot date and address family in the message so tokens cannot be joined across retention windows by accident.

Pseudocode:

message = pilot_day || address_family || canonical_address_bytes
exit_token = HMAC-SHA256(daily_secret, message)
store(truncate(exit_token, 128_bits))
Enter fullscreen mode Exit fullscreen mode

The daily secret belongs in a secret manager, not in the event record. Delete or rotate it according to the approved retention plan. A keyed construction matters: ordinary hashing is vulnerable to guessing across the small address space.

3. Build nested outcome sets

For each cohort, create four sets:

observed  = unique(exit_token where gateway_connection_attempted)
healthy   = unique(exit_token where controlled_transport_check_passed)
usable    = unique(exit_token where authorized_workflow_valid)
surviving = unique(exit_token where session_requirement_met)
Enter fullscreen mode Exit fullscreen mode

Then calculate:

usable_yield = size(usable) / size(observed)
Enter fullscreen mode Exit fullscreen mode

Do not count retries as new supply. The uniqueness key is the exit token inside the chosen time window.

4. Measure replacement, not just failure

When an exit becomes unusable, start a replacement timer. Stop only when the gateway supplies a different token that completes the required layer.

replacement_ms = usable_replacement_at - failure_detected_at
Enter fullscreen mode Exit fullscreen mode

Report p50, p90, and p95. A mean can look healthy while a small but important tail causes long production stalls.

5. Create a survival curve

Group exits by first_seen_at. At each required duration, count the cohort members that still meet the correct definition of usable.

S(t) = usable exits from cohort at time t / exits in cohort at t0
Enter fullscreen mode Exit fullscreen mode

For sticky sessions, “usable” should include application state, expected market, and network class—not merely an unchanged IP token.

6. Detect recycling and drift 

recycling_rate = repeated_successful_assignments / successful_assignments
Enter fullscreen mode Exit fullscreen mode

Compute it in rolling 5-, 15-, and 60-minute windows. Also record transitions in market, address family, and privacy-minimized ASN class. Keep rotating and sticky endpoints in separate cohorts; their expected distributions are fundamentally different.

7. Keep three test layers separate

  • Gateway supply: did the intended proxy deliver an exit with no direct fallback?
  • Controlled transport: did TLS, latency, and response integrity pass on an endpoint you own?
  • Authorized workflow: did the real destination return valid content under its rules?

Mixing these layers turns destination throttling into a proxy-quality failure, or transport success into false business success.

Acceptance gates

Fix thresholds before looking at results. A market-specific gate might require usable yield above 85%, p95 replacement below 20 seconds, 90% sticky-session survival at 30 minutes, market mismatch below 3%, and zero direct fallback.

Those are examples, not universal defaults. Segment by market, IPv4/IPv6, time window, and mode, and use confidence intervals for small cohorts.

For the full trilingual procurement guide and checklist, see 98IP: https://en.98ip.com/?k=dev

Test only authorized systems and data flows. Respect terms, access controls, consent, privacy rules, rate limits, and automated-traffic identification requirements. Rotation is not permission to evade enforcement.

Top comments (0)