DEV Community

proxymint
proxymint

Posted on

curl through a proxy can exit 0 when the login fails

#curl #bash #devops #networking

Small thing that probably bites a lot of cron jobs out there: curl through a proxy, wrong password, and $? is 0.

No crash, no stderr. The output file just has an empty 407 in it where the page should be, and whatever reads that file next is going to have a bad day.

I wanted to know exactly when this happens, so I wrote a fake proxy that refuses everyone and pointed curl 8.7.1 at it.

# p407.py - a fake proxy that refuses every login
import http.server, socketserver

class H(http.server.BaseHTTPRequestHandler):
    def refuse(self):
        self.send_response(407)
        self.send_header("Proxy-Authenticate", 'Basic realm="p"')
        self.send_header("Content-Length", "0")
        self.end_headers()
    do_GET = do_CONNECT = do_HEAD = refuse

socketserver.TCPServer(("127.0.0.1", 18407), H).serve_forever()
Enter fullscreen mode Exit fullscreen mode

http:// target: exit 0 

curl -s -o out.html -w '%{http_code}\n' -x http://user:pass@127.0.0.1:18407 http://example.com/
echo "exit $?"
Enter fullscreen mode Exit fullscreen mode 
407
exit 0
Enter fullscreen mode Exit fullscreen mode

Yep. With a plain http URL curl hands the whole request to the proxy, the proxy says 407, and that comes back like any other response. curl got an answer, job done. Bad password, an @ in the password nobody encoded, plan ran out, doesn't matter, your script sees a clean exit.

-f fixes it 

curl -sf -o out.html -x http://user:pass@127.0.0.1:18407 http://example.com/
echo "exit $?"
Enter fullscreen mode Exit fullscreen mode 
exit 22
Enter fullscreen mode Exit fullscreen mode

-f tells curl that 400 and up is a failure, so you get 22. --fail-with-body does the same but keeps the body, handy when you want to see what the proxy actually said.

https:// target: fails anyway, different number 

curl -sS -o /dev/null -x http://user:pass@127.0.0.1:18407 https://example.com/
echo "exit $?"
Enter fullscreen mode Exit fullscreen mode 
curl: (56) CONNECT tunnel failed, response 407
exit 56
Enter fullscreen mode Exit fullscreen mode

Here curl has to ask the proxy for a tunnel first (CONNECT), and the 407 shows up before TLS even starts, so there's nothing to pretend about. 56 on 8.7.1. We got 7 for the same thing on 8.21.0, so if your script matches exact numbers, don't. Non-zero means it broke.

Honestly the https case isn't the scary one. The scary one is the cheap health check or the old scraper hitting an http endpoint, because that's where it fails quietly for days.

The fix I'd use

Check the status code too, not only the exit code:

code=$(curl -s -o out.html -w '%{http_code}' -x "$PROXY" "$URL")
rc=$?
if [ "$rc" -ne 0 ] || [ "$code" -ge 400 ]; then
  echo "failed: exit=$rc status=$code" >&2
  exit 1
fi
Enter fullscreen mode Exit fullscreen mode

Or just -f plus set -e if you like it short.

Two neighbours of this bug:

  • SOCKS5 can't give you a 407 at all, there's no HTTP in between. A bad login there is curl: (97) User was rejected by the SOCKS5 server. Other number, same problem.
  • p@ss in a proxy URL has to be p%40ss. Otherwise curl cuts the URL at the wrong @, sends a username the proxy has never heard of, and you're staring at a 407 for a password you know is right.

We wrote up the rest of curl-through-a-proxy (socks5 vs socks5h and DNS leaks, --noproxy, timing each hop with -w, a five-command check that tells you which layer broke) here: curl proxy: -x, auth, SOCKS5, timing and exit codes.

Disclosure: I work on proxymint, a proxy service. None of the above depends on whose proxy it is.

Top comments (0)