DEV Community

RoamProxy
RoamProxy

Posted on

Your proxy connected. That doesn't mean it hid your IP.

#python #webscraping #security #devops

Every proxy checker I've come across answers one question: did the connection open?

That's the least interesting thing about a proxy. A proxy can connect perfectly and still hand your real IP address to the destination server in a header. If you're scraping, testing geo-restricted behaviour, or just trying not to be identified, "it connected" tells you almost nothing.

Here's how to actually check, with code you can run.

Three ways a proxy gives you away

1. It forwards your real IP in a header

The most direct leak. The proxy adds your address to the request:

GET /whatever HTTP/1.1
Host: example.com
X-Forwarded-For: 203.0.113.44     <- your actual IP
Enter fullscreen mode Exit fullscreen mode

X-Forwarded-For is the common one, but X-Real-IP and the RFC 7239 Forwarded header do the same job. This is normal, correct behaviour for a reverse proxy sitting in front of your server — and completely wrong for a forward proxy you're using to not be identified.

A proxy that does this is called transparent. The destination sees both the proxy's IP and yours.

2. It announces that it's a proxy

Sometimes your IP is safely hidden, but the request still carries:

Via: 1.1 squid-cache
Proxy-Connection: keep-alive
Enter fullscreen mode Exit fullscreen mode

Your identity is intact, but the server knows the traffic is proxied. Plenty of anti-bot systems treat that alone as reason enough to block, rate-limit, or serve you different content. So it matters even though nothing personal leaked.

3. The exit is not where it claims to be

A proxy sold as Frankfurt that egresses in Singapore isn't just a labelling problem — it breaks anything geo-dependent, and the latency will be nothing like what you planned for.

Checking it properly

The key insight: you need a baseline. Without knowing your own IP first, you can't tell "the proxy leaked my address" apart from "there happens to be an IP in this header."

import httpx

# Baseline: who am I without a proxy?
direct_ip = httpx.get("https://api.ipify.org", timeout=10).text.strip()

PROXY = "http://user:pass@proxy.example.com:8080"

with httpx.Client(proxy=PROXY, timeout=10) as client:
    exit_ip = client.get("https://api.ipify.org").text.strip()
    seen_headers = client.get("https://httpbin.org/headers").json()["headers"]

print(f"me:   {direct_ip}")
print(f"exit: {exit_ip}")
print(f"headers the server saw: {seen_headers}")
Enter fullscreen mode Exit fullscreen mode

Now grade it:

PROXY_HEADERS = (
    "via", "x-forwarded-for", "x-real-ip",
    "forwarded", "proxy-connection",
)

def grade(headers: dict, direct_ip: str | None) -> str:
    lowered = {k.lower(): str(v) for k, v in headers.items()}

    # Our real IP showed up somewhere in what the server received.
    if direct_ip and any(direct_ip in v for v in lowered.values()):
        return "transparent"

    # Identity is safe, but the request is visibly proxied.
    if any(h in lowered for h in PROXY_HEADERS):
        return "anonymous"

    return "elite"
Enter fullscreen mode Exit fullscreen mode

Three outcomes:

Grade What it means
transparent Your real IP is visible through the proxy. Useless for anonymity.
anonymous Real IP hidden, but headers announce a proxy is in use.
elite Looks like an ordinary direct request.

The part that's easy to get wrong

Note the direct_ip and ... guard. If fetching your baseline fails — no network, endpoint down, rate limited — direct_ip is None, and a naive implementation that does if direct_ip in value will either crash or silently match nothing.

Either way you must not report transparent. Claiming a proxy is safe when you simply failed to check is the one error mode that actually hurts someone. Degrade to anonymous, which is what the header evidence alone supports.

Same reasoning applies to the header check being case-insensitive. Servers and proxies are inconsistent about casing, and "Via" vs "via" deciding whether you flag a leak is not a distinction you want in security-adjacent code.

Doing this to a whole list

One proxy at a time is fine for debugging. For a list of a few hundred you want concurrency, timeouts, and output you can pipe somewhere.

I put the above into a single-file CLI called proxyprobe — MIT, one dependency:

pip install "httpx[socks]"
curl -O https://raw.githubusercontent.com/roamproxy/proxyprobe/main/proxyprobe.py
python proxyprobe.py proxies.txt --geo
Enter fullscreen mode Exit fullscreen mode 
PROXY                              OK   MS    EXIT IP        ANONYMITY    LOCATION
---------------------------------  ---  ----  -------------  -----------  -----------------
http://user:***@gw.example.com:80  yes  312   203.0.113.44   elite        Tokyo Japan
socks5://198.51.100.7:1080         yes  1180  198.51.100.7   transparent  Frankfurt Germany
http://10.0.0.9:3128               no   -     -              -

2/3 working  |  312ms median  |  1 elite, 1 transparent
Enter fullscreen mode Exit fullscreen mode

It exits 0 if anything worked and 1 if nothing did, so it drops into CI:

proxyprobe proxies.txt --working-only --json > alive.json || echo "all proxies down"
Enter fullscreen mode Exit fullscreen mode

Two implementation details worth stealing even if you write your own:

Mask passwords greedily. Proxy credentials routinely contain @ and :. A lazy regex like ://([^:/@]+):([^@]+)@ stops at the first @ and leaves the tail of the password in your output:

http://u:p@ss:word@host:80   ->   http://u:***@ss:word@host:80   # leaked
Enter fullscreen mode Exit fullscreen mode

Match greedily to the last @ instead. A unit test caught this one for me, which is a good argument for unit-testing the boring string function that touches secrets.

Make the endpoints configurable. Anything hardcoded means someone routes their entire proxy list through a third party they never chose. --echo-url and --headers-url take about four lines and remove that objection entirely.

What this doesn't cover

This is an HTTP-layer check. If you're driving a real browser, there's a separate set of leaks that live above it — WebRTC handing out local candidates, DNS resolving outside the tunnel, timezone and locale disagreeing with the exit IP. None of that shows up in request headers, and none of it is detectable with the code above.

Worth knowing where the boundary is: passing this check means the proxy isn't betraying you at the HTTP layer. It doesn't mean a browser sitting on top of it is quiet.

Disclosure: I work on Roam, a proxy provider. proxyprobe works with proxies from anyone and has no vendor lock-in — we open-sourced it because we were writing this script over and over internally. If you use it against a competitor's proxies and find a bug, I'd genuinely like the issue.

Top comments (0)