DEV Community

upordown.online
upordown.online

Posted on Edited on

How to read curl -v output to find where a request fails

#curl #debugging #networking #webdev

When a site will not load, the browser usually shows one vague error. curl -v shows you every step of the request instead: name lookup, TCP connect, TLS handshake, the request line, and the response. Once you know how to read that trace, you can tell in a few seconds whether the problem is DNS, the network path, the certificate, or the application itself.

The command 

curl -v -o /dev/null https://example.com/
Enter fullscreen mode Exit fullscreen mode

-v prints the conversation to stderr, and -o /dev/null throws away the body so the trace stays readable. Add --max-time 15 if you don't want to wait forever on a hung server.

Step 1: name resolution

The first lines look like this:

* Host example.com:443 was resolved.
* IPv6: (none)
* IPv4: 93.184.215.14
Enter fullscreen mode Exit fullscreen mode

If the trace stops here with Could not resolve host, the request never left your machine. The domain has no DNS record, your resolver is failing, or the name is mistyped. Check it with a different resolver:

dig +short example.com @1.1.1.1
dig +short example.com @8.8.8.8
Enter fullscreen mode Exit fullscreen mode

If public resolvers answer and yours does not, the problem is local (router, VPN, corporate DNS). If nobody answers, the domain's DNS is broken or expired.

Step 2: the TCP connect 

*   Trying 93.184.215.14:443...
* Connected to example.com (93.184.215.14) port 443
Enter fullscreen mode Exit fullscreen mode

Two failures are common here:

  • Connection refused: the host answered, but nothing is listening on that port. The web server process is down, or a firewall actively rejects you.
  • Connection timed out / Failed to connect ... after 15000 ms: packets go out and nothing comes back. That points to a firewall that silently drops traffic, a dead host, or a routing problem.

Refused is fast and timed out is slow, and that difference alone tells you a lot.

Step 3: the TLS handshake 

* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* Server certificate:
*  subject: CN=example.com
*  expire date: Jan 15 23:59:59 2027 GMT
*  subjectAltName: host "example.com" matched cert's "example.com"
*  SSL certificate verify ok.
Enter fullscreen mode Exit fullscreen mode

Read three things: the expire date, whether the subjectAltName matched, and whether verification succeeded. Typical errors:

  • SSL certificate problem: certificate has expired
  • no alternative certificate subject name matches target host name (the certificate is for a different domain, often after a CDN or host change)
  • unable to get local issuer certificate (the server is not sending its intermediate certificate)

To isolate a TLS problem, retry with -k. If the page loads with -k and fails without it, the server is fine and the certificate is the issue. Don't leave -k in scripts.

Step 4: the request and response

Lines starting with > are what curl sent, and lines starting with < are what came back:

> GET / HTTP/2
> Host: example.com
> User-Agent: curl/8.5.0
>
< HTTP/2 503
< server: nginx
< retry-after: 120
Enter fullscreen mode Exit fullscreen mode

Getting this far means DNS, network and TLS all work. Now the status code tells the story. A 5xx means the server or something behind it failed. A 403 may mean a firewall or bot filter does not like your client. Retry with a browser user agent to check:

curl -v -o /dev/null -A 'Mozilla/5.0' https://example.com/
Enter fullscreen mode Exit fullscreen mode

If that returns 200, the site is up and is filtering by user agent.

Step 5: redirects

By default curl does not follow redirects. Add -L and you will see each hop:

< HTTP/2 301
< location: https://www.example.com/
Enter fullscreen mode Exit fullscreen mode

A trace that bounces between the same two URLs is a redirect loop, usually caused by conflicting http-to-https or www rules in two places (for example the CDN and the origin).

Timing each phase

For slow sites, -w prints how long each phase took:

curl -o /dev/null -s -w 'dns %{time_namelookup}\nconnect %{time_connect}\ntls %{time_appconnect}\nfirst byte %{time_starttransfer}\ntotal %{time_total}\n' https://example.com/
Enter fullscreen mode Exit fullscreen mode

The values are cumulative. A big jump between tls and first byte means the server is slow to generate the page; a big dns value means slow name resolution.

Is it you or them?

A clean trace from your machine still doesn't prove the site works for everyone, and a failing trace doesn't prove it's down. Your network, ISP or region might be the variable. Running the same check from somewhere else is the quickest way to know, which is what upordown.online does: it tests DNS, connect, TLS and the first bytes of the page from its own server and shows what other visitors report.

Cheat sheet

Where the trace stops Likely cause
Could not resolve host DNS
Connection refused Service down or port closed
Connection timed out Firewall drop, host down, routing
SSL certificate problem Expired, wrong name, missing intermediate
HTTP 5xx Application or upstream failure
HTTP 403 with curl, 200 with browser UA Bot filtering

Related reading

Top comments (0)