Fixing WebSocket Connection Issues with Nginx Reverse Proxy

Introduction

Deploying an application with Nginx as a reverse proxy usually works smoothly—until you encounter issues with WebSocket connections, such as webhooks failing to connect. If you're seeing errors like "Unexpected server response: 400", this post will guide you through diagnosing and resolving WebSocket connection problems. We'll cover the necessary Nginx configuration for WebSocket support, common issues, and their solutions.

Managing Nginx Configuration for WebSocket Connections

WebSocket connections require specific configurations in Nginx to function correctly. Misconfigurations often lead to errors like the dreaded 400 response. Below, we explore the potential causes of WebSocket failures and provide actionable solutions.

Potential Causes and Solutions

1. Incorrect Server-Side WebSocket Configuration
The server must be configured to handle WebSocket connections at the designated endpoint. This involves supporting the Upgrade and Connection headers during the WebSocket handshake. Here's an example of a proper Nginx configuration for WebSocket support:

server {
    listen 80;
    server_name your-domain.com;
    location /socket.io/ { 
         proxy_pass http://backend_server; 
         proxy_http_version 1.1; 
         proxy_set_header Upgrade $http_upgrade; 
         proxy_set_header Connection "Upgrade"; 
         proxy_set_header Host $host; 
         proxy_set_header Sec-WebSocket-Key $http_sec_websocket_key; 
         proxy_set_header Sec-WebSocket-Version $http_sec_websocket_version;
         proxy_set_header Sec-WebSocket-Extensions $http_sec_websocket_extensions;

   # Optional: Increase timeouts for long-lived WebSocket connections
        proxy_read_timeout 86400s;
        proxy_send_timeout 86400s;
    }
}

Key Points:

• Replace your-domain.com with your actual domain.
• Update backend_server to the address of your application server (e.g., http://localhost:3000).
• The proxy_read_timeout and proxy_send_timeout settings prevent connections from timing out during long-running WebSocket sessions.
• After updating the configuration, reload Nginx to apply the changes:

sudo nginx -s reload

2. Invalid WebSocket Request Format
The client must send a properly formatted WebSocket handshake request, including the correct Upgrade and Connection headers. If these are missing or incorrect, the server may reject the request with a 400 error.

Solution:

• Verify the client-side WebSocket implementation (e.g., using libraries like Socket.IO).
• Ensure the request includes headers like Sec-WebSocket-Key and Sec-WebSocket-Version.
• Test the handshake using tools like Postman or wscat to confirm the request format.

3. Firewall or Proxy Blocking WebSocket Traffic
Firewalls, Web Application Firewalls (WAFs), or other proxies between the client and server may block WebSocket traffic, mistaking it for malicious activity.

Solution:

• Check your firewall rules to ensure WebSocket ports (e.g., 80 for ws:// or 443 for wss://) are open.
• If using a WAF (e.g., Cloudflare), configure it to allow WebSocket traffic by enabling WebSocket support in its settings.
• Test connectivity by temporarily disabling the firewall or WAF to isolate the issue.

4. WebSocket Version Mismatch
A mismatch between the client and server WebSocket protocol versions (e.g., Socket.IO client and server versions) can cause handshake failures.

Solution:

• Ensure the client and server use compatible WebSocket library versions.
• For Socket.IO, confirm that both client and server are running the same major version (e.g., Socket.IO v4.x).
• Update libraries if necessary using package managers like npm or pip.

5. SSL/TLS Configuration Issues
For secure WebSocket connections (wss://), SSL/TLS must be configured correctly. Invalid or expired SSL certificates can cause connection failures.

Reload Nginx after making changes:

sudo nginx -s reload