DEV Community

Cover image for http_proxy environment variable: what each tool actually reads
proxymint
proxymint

Posted on Originally published at proxymint.com

http_proxy environment variable: what each tool actually reads

Originally published on proxymint.

How http_proxy, HTTPS_PROXY, ALL_PROXY and NO_PROXY work in curl, wget, Python, Go, Node, Ruby and Java: case rules, NO_PROXY matching, debugging.

TL;DR

  • Export both cases with the same value. curl and wget only read lowercase http_proxy, Go prefers HTTP_PROXY when both exist, so two values that drift apart send two tools to two different proxies.
  • Set https_proxy too, or HTTPS goes direct. The tool picks the variable by the site's scheme, and the page still loads without the proxy, which is why this one goes unnoticed for weeks.
  • Keep the value http:// even in HTTPS_PROXY. It looks wrong, but the https in the name is about the sites you visit; your client sends CONNECT to the proxy over plain HTTP and the site's TLS runs inside that tunnel.
  • Node and Java need extra steps. Node's fetch ignores the variables unless NODE_USE_ENV_PROXY=1 is set, and Java never reads them at all; it wants -Dhttps.proxyHost and friends.

Key takeaways

  • Run env | grep -i proxy before touching code. An uppercase-only value or two different values explains most "it ignores my proxy" tickets.
  • Write no_proxy as example.com,.example.com. It looks redundant, but that pair bypassed the domain and its subdomains in every tested tool, while *.example.com and CIDR ranges each failed in at least three.
  • Set no_proxy in both cases. wget only reads the lowercase name and will send internal hosts through the proxy otherwise.
  • Put the variables where the job runs. systemd units, sudo and cron do not load your .bashrc, so a proxy that works in your terminal can be missing from the service.
  • Point the variables at a logging proxy when in doubt. A ten-line listener shows whether a tool sent GET, CONNECT or nothing, which ends the guessing.

Read the full guide on proxymint, with the commands, tables and sources.

Top comments (0)