DEV Community

Rasika Dangamuwa
Rasika Dangamuwa

Posted on

Why SSL/TLS Certificates Break in Production: 5 Real-World Traps Every Engineer Hits

#webdev #programming #security #devops

Every engineer has experienced that sinking feeling: an alert fires at 2 a.m., or users report seeing "Your connection is not private" error screens. You check your reverse proxy dashboard and everything looks green. You load the site in desktop Chrome, and the padlock icon is completely fine.

So why are production requests failing?

SSL/TLS management feels solved until edge routing, CDNs, automated renewals, and multi-tenant architectures collide. Subtle certificate configuration errors often pass manual desktop checks while silently breaking mobile apps, API clients, and webhooks. Here are five real-world SSL/TLS certificate traps that routinely break production systems, along with practical fixes.

1. The Missing Intermediate Certificate (Incomplete Chain)

This is the single most common SSL bug in web deployments.

The Trap: When a Certificate Authority (CA) issues an SSL certificate, it signs your leaf certificate using an intermediate CA certificate, which traces back to a trusted root CA. Web servers (Nginx, Apache, HAProxy) must serve both your leaf certificate and the intermediate bundle (fullchain.pem).

If you configure only cert.pem instead of fullchain.pem:

# Broken: serves only leaf certificate
ssl_certificate /etc/letsencrypt/live/example.com/cert.pem;

# Correct: serves leaf + intermediate chain
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
Enter fullscreen mode Exit fullscreen mode

Desktop browsers often mask this because they cache intermediate certificates from previous sessions or fetch missing intermediates via AIA fetching. But mobile browsers, curl, Python requests, and API clients do not perform AIA caching. They fail immediately:

SSL certificate problem: unable to get local issuer certificate
Enter fullscreen mode Exit fullscreen mode

When diagnosing connection drops across mobile apps and API clients, inspecting your live endpoints with Nutilz SSL Checker or running openssl s_client -connect example.com:443 -servername example.com exposes whether intermediate CAs are missing from your served bundle.

2. Wildcard Certificates Do Not Cover Nested Subdomains

Wildcard certificates (*.example.com) simplify certificate management across multiple services.

The Trap: Under RFC 6125, the asterisk wildcard matches exactly one domain component. It does not cross dots to match nested subdomains:

  • *.example.com matches api.example.com and web.example.com.
  • *.example.com fails for v1.api.example.com or staging.auth.example.com.
  • *.example.com also does not match the apex domain example.com.

Deploying a microservice to a two-level subdomain breaks connections instantly with ERR_CERT_COMMON_NAME_INVALID.

The Fix: Include explicit Subject Alternative Names (SANs) for nested subdomain tiers in your CSR:

[ alt_names ]
DNS.1 = example.com
DNS.2 = *.example.com
DNS.3 = *.api.example.com
Enter fullscreen mode Exit fullscreen mode

3. Silent ACME / Certbot Renewal Failures

Most production environments use ACME clients (Certbot, Caddy, Traefik) to renew 90-day certificates automatically around day 60.

The Trap: Automated renewals work until a routine infrastructure update breaks challenge validation:

  1. An Nginx rewrite rule redirects all HTTP traffic to HTTPS or strips the /.well-known/acme-challenge/ path.
  2. A CDN or WAF update blocks ACME validation bots.
  3. A systemd timer or cron job fails due to rotated API keys or file permission changes.

Because existing certificates remain valid for roughly 30 days after the first failed renewal, the failure goes unnoticed until the certificate hard-expires.

The Fix:

  • Run renewal dry-runs after ingress changes: certbot renew --dry-run
  • Alert based on actual certificate validity remaining (< 20 days) rather than cron execution status.

4. SNI Routing Mismatches on Shared Ingress Proxies

Server Name Indication (SNI) tells the server which hostname the client wants before the TLS handshake completes.

The Trap: When multiple domains share a single IP or load balancer, the proxy uses SNI to present the correct certificate. If an internal service, health check probe, or legacy backend connects without specifying SNI or connects via direct IP (https://198.51.100.42), the server falls back to its default certificate. If that default belongs to a different domain, the handshake fails with a hostname mismatch.

The Fix:
Configure Nginx to reject handshakes that lack valid matching SNI hosts:

server {
    listen 443 ssl default_server;
    ssl_reject_handshake on; # Nginx 1.19.4+
}
Enter fullscreen mode Exit fullscreen mode

5. Overly Strict TLS 1.3 Negotiation

Modern security baselines rightly deprecate TLS 1.0 and 1.1.

The Trap: In an effort to harden configurations, teams sometimes disable TLS 1.2 entirely (ssl_protocols TLSv1.3;). While modern browsers handle TLS 1.3 seamlessly, legacy payment gateways, webhook dispatchers, and IoT clients often rely on TLS 1.2 with specific ECDHE ciphers. Webhooks fail silently with TLS handshake errors.

The Fix: Support both TLS 1.2 and TLS 1.3 using Mozilla’s Intermediate configuration:

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;
Enter fullscreen mode Exit fullscreen mode

Production TLS Checklist

  1. Serve full chains: Configure fullchain.pem, never just leaf cert.pem.
  2. Verify SAN hierarchy: Ensure wildcards match exact subdomain depth.
  3. Monitor days remaining: Alert on expiration dates independently of renewal scripts.
  4. Enforce SNI strictly: Use ssl_reject_handshake on to avoid leaking mismatched certificates.
  5. Inspect live certificates: Audit public endpoints using Nutilz SSL Checker or openssl s_client to confirm chain completeness, validity periods, and cipher suites across all public services.

Top comments (0)