DEV Community

ProxyMaster
ProxyMaster

Posted on

cURL and Proxies: Every Flag You Actually Need

WinGate private IPv4 and SOCKS5 proxies, free 2 hour test

Before a proxy goes into your scraper, you test it with curl. Everyone does. And curl is the fastest way to answer the only question that matters at that stage: is it the proxy or is it my code? But curl has a pile of proxy flags, and using the wrong one sends you chasing phantom problems. Here are the flags you actually need and what each one really does.

The basic proxy flag

-x (or --proxy) sets the proxy for the request. It takes a full URL, and the scheme decides the proxy type.

curl -x http://proxy.host:8080 https://example.com
Enter fullscreen mode Exit fullscreen mode

If your proxy is SOCKS5, say so in the scheme:

curl -x socks5h://proxy.host:1080 https://example.com
Enter fullscreen mode Exit fullscreen mode

The h in socks5h matters: it tells curl to resolve DNS at the proxy, not locally. Without it, your DNS lookups leak from your real location, which defeats half the point of the proxy.

Authentication

-U (or --proxy-user) passes proxy credentials, kept separate from -u which is for the target site.

curl -x http://proxy.host:8080 -U user:pass https://example.com
Enter fullscreen mode Exit fullscreen mode

Do not confuse -U and -u. -u authenticates to the website, -U authenticates to the proxy. Mixing them up produces a 407 that looks like a credential problem when the credentials are actually fine, just aimed at the wrong layer.

Flags that tell you what is really happening

When a proxy request misbehaves, these turn guesswork into facts.

  • -v shows the full handshake, including the CONNECT tunnel for HTTPS and the proxy's response. This is where you see a 407 or a tunnel failure directly.
  • -w "%{http_code} %{time_total}s\n" prints the status and total time, so you can measure latency and spot throttling.
  • -o /dev/null -s throws away the body and silences the progress bar when you only care about the status.
  • --connect-timeout 10 stops a dead proxy from hanging your test forever.

A one-liner that checks a proxy cleanly:

curl -x http://user:pass@proxy.host:8080 \
  -o /dev/null -s -w "%{http_code} %{time_total}s\n" \
  --connect-timeout 10 https://httpbin.org/ip
Enter fullscreen mode Exit fullscreen mode

Confirm the exit address

The single most useful test is hitting an endpoint that echoes your IP, so you confirm the request actually left through the proxy and see which exit you got.

curl -x http://user:pass@proxy.host:8080 -s https://httpbin.org/ip
Enter fullscreen mode Exit fullscreen mode

Run it a few times against a rotating endpoint and you should see the address change. If it does not, your rotation is not working, or curl is reusing a connection, and that is worth knowing before you build a whole scraper on top.

From curl to a real pool

curl proves a single request works. A real scrape needs a pool that stays healthy across thousands of them. WinGate provides private IPv4 proxies with SOCKS5 that respond to exactly these curl flags, so what you test is what you run. The rotating endpoint changes the exit each request, which you can watch happen with the httpbin test above, traffic is unlimited, and it supports HTTP, HTTPS, and SOCKS5 so the same flags carry over. There is a free 2 hour test, so you can curl a real endpoint and confirm the rotation and latency before writing a line of scraper code.

An honest note: a clean curl result means the proxy works right now for one request, it does not promise a site will not block you at volume, and it does not exempt you from the site's terms. curl is a diagnostic, not a guarantee. What it gives you is a fast, honest answer to whether the problem is the proxy or your code.

The takeaway: -x for the proxy with the right scheme, socks5h to route DNS, -U for proxy auth not -u, and -v plus -w to see what is really happening. Master those five and you can diagnose any proxy in seconds instead of guessing.

Top comments (0)