DEV Community

ProxyMaster
ProxyMaster

Posted on Edited on

Handling Proxy Authentication in Node.js with axios and got

#proxy #webscraping #node #javascript

WinGate private IPv4 and SOCKS5 proxies, free 2 hour test

Proxy authentication is one of those things that looks trivial until it silently breaks your whole scraper. You add a proxy, the requests go out, and everything works on your machine. Then you deploy, the credentials get mangled by a URL parser, and every request comes back 407 Proxy Authentication Required. This post covers how to pass proxy credentials correctly in the two Node.js HTTP clients most projects reach for, axios and got, and how to keep them working under rotation.

Why 407 shows up

A proxy that requires a username and password rejects any request that does not present them, and it does so with a 407 status. The tricky part is that credentials travel in different places depending on the client. Some read them from the proxy URL, some want a separate auth object, and some need a Proxy-Authorization header you build yourself. Get the placement wrong and the proxy never sees your credentials, so it refuses the connection before your target is ever contacted.

axios with a proxy

axios accepts a proxy object with host, port, and auth. Passing credentials as a structured object avoids the URL-encoding traps that break passwords containing special characters.

const axios = require("axios");

const client = axios.create({
  proxy: {
    host: "proxy.host",
    port: 8080,
    auth: { username: "user", password: "pass" },
  },
  timeout: 20000,
});

const res = await client.get("https://example.com");
console.log(res.status);
Enter fullscreen mode Exit fullscreen mode

For HTTPS targets, many setups are more reliable with an explicit proxy agent such as https-proxy-agent, passed as httpsAgent, because axios then tunnels through the proxy with CONNECT instead of relying on its built-in handling.

got with a proxy

got does not take a proxy option directly. You give it an agent built from a proxy library, which keeps credentials in the proxy URL where the agent expects them.

import got from "got";
import { HttpsProxyAgent } from "hpagent";

const agent = new HttpsProxyAgent({
  proxy: "http://user:pass@proxy.host:8080",
});

const res = await got("https://example.com", {
  agent: { https: agent },
  timeout: { request: 20000 },
});
console.log(res.statusCode);
Enter fullscreen mode Exit fullscreen mode

If your password contains characters like @ or :, URL-encode them before putting them in the proxy string, otherwise the parser will split the URL in the wrong place and you are back to 407.

Keeping it working under rotation

Authentication and rotation interact. With a rotating endpoint you usually keep the same credentials and the same host and port, and the pool changes the exit address behind the scenes. That means you configure auth once and never touch it again while addresses cycle underneath. A few habits keep this stable:

  • One credential set, one endpoint. Point every request at the same authenticated proxy URL and let the pool rotate. Do not rebuild the auth per request.
  • Retry on a fresh address, not the same one. A single failed request should be retried through the rotating endpoint so it lands on a new exit rather than pounding the address that just failed.
  • Prefer private addresses. Shared proxies fail intermittently because someone else already burned them, which looks exactly like an auth bug and wastes hours of debugging.

This is where the choice of provider matters more than the client code. WinGate offers private proxies with SOCKS5 and standard user and password auth, so the credentials you set in axios or got stay valid while the rotating pool changes the exit for you. The addresses are private rather than shared, traffic is unlimited, and the same endpoint speaks HTTP, HTTPS, and SOCKS5, so it drops into either client without special cases.

An honest note: correct auth setup stops the 407 errors, but it does not make a scraper polite by itself. You still need sensible pacing and retries, and a proxy does not exempt you from a site's terms. There is a free 2 hour test, so wire your real credentials into axios or got, run a batch, and confirm you get clean 200 responses before you commit.

Related reading

The short version: 407 is almost always a placement problem, not a permissions problem. Put the credentials where your client expects them, keep one authenticated endpoint under rotation, retry on fresh addresses, and proxy auth stops being the thing that quietly breaks your deploy.

Top comments (0)