Troubleshooting and Resolving WebSocket 403 Forbidden Errors Through CDN (wstest.example.com)
)##
In a Kubernetes deployment using NGINX Ingress and a Go-based WebSocket service (go-push
), we encountered a problem after routing traffic through a CDN provider (Funnell) using the following domain:
wss://wstest.example.com/push/ws?token=xxx
The backend service listens on port 8081
and is designed to handle WebSocket connections.
The Problem
When connecting to the WebSocket endpoint via a browser or tools like Postman, we consistently received:
403 Forbidden
With minimal response headers::
Date: ...
Content-Length: 0
No requests were seen in the backend service logs, indicating the request never reached the application layer.
Investigation Steps
1. Check Ingress Configuration
The Kubernetes Ingress was configured with the following annotations:
annotations:
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
nginx.ingress.kubernetes.io/connection-proxy-header: keep-alive
nginx.ingress.kubernetes.io/use-forwarded-headers: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header X-Request-From gateway;
The routing rule was defined as:
- path: /push(/|$)(.*)
pathType: ImplementationSpecific
backend:
service:
name: go-push
port:
number: 8081
This configuration rewrites /push/ws
to /ws
and forwards it to the backend service.
2. Bypass the CDN to Verify Ingress Functionality
I used curl to simulate a WebSocket handshake request:
curl -i -N \
-H "Connection: Upgrade" \
-H "Upgrade: websocket" \
-H "Sec-WebSocket-Version: 13" \
-H "Sec-WebSocket-Key: test123==" \
-H "Host: wstest.example.com" \
http://<Ingress-IP>:<Port>/push/ws
I received a 101 Switching Protocols
response, confirming that both Ingress and the backend were functioning correctly. The issue was clearly upstream—in the CDN layer.
3. Identify CDN as the Source of the 403
Based on the response and headers, we confirmed the 403 Forbidden
was returned by the CDN (Funnell), not by Kubernetes or the backend service. Likely causes included:
WebSocket support not enabled
The
/push/ws
path not allowed in origin routingRequired WebSocket headers being stripped
Solution (Funnell CDN Configuration)
To resolve this issue, we made the following changes in the Funnell CDN management console for the domain wstest.example.com
:
Enable WebSocket Support
- Activate the WebSocket support or Connection Upgrade option in the CDN settings
Allow the WebSocket Path in Origin Routing
Explicitly allow paths like
/push/.*
in the origin path whitelistIf the default routing only allows
/api
or/static
, custom paths will be blocked with 403
Preserve WebSocket Headers
- Ensure the following headers are forwarded without modification:
Header | Expected Value |
---|---|
Connection |
Upgrade |
Upgrade |
websocket |
Sec-WebSocket-* |
As-is |
Successful Validation
After updating the CDN settings, WebSocket handshake requests to:
wss://wstest.example.com/push/ws?token=xxx
succeeded with a valid response:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
And the backend logs confirmed the connection was received and processed correctly.
Conclusion
When WebSocket requests routed through a CDN return 403 Forbidden, the root cause is often not in the backend or Ingress but in the CDN's security and protocol handling rules.
To ensure WebSocket connections work correctly through a CDN:
Enable WebSocket protocol support
Allow the WebSocket path in origin routing
Preserve critical handshake headers (Upgrade, Connection)
Use proper rewrite and header forwarding rules in Ingress
With these in place, it's possible to enjoy both CDN performance and stable WebSocket connectivity.
Top comments (0)