Key takeaways
- A reverse HTTP proxy over WebSocket is a type of proxy server which uses the WebSocket protocol as a "tunnel" to pass TCP communication from server to client.
- In Go project, gorilla/websocket is widely used to implement WebSocket.
- WebSocket is designed to work over HTTP. To achieve compatibility with HTTP, the WebSocket handshake uses the HTTP Upgrade header in order to change from the HTTP protocol to the WebSocket protocol.
- Authentication of WebSocket is a difficult design problem, and there are many options.
Reverse HTTP proxy over WebSocket
There are two types of proxies, forward proxy and reverse proxy.
A forward proxy (or gateway, or tunnel, or just "proxy") provides proxy services to a client or a group of clients. It allow users to hide their IP address while accessing a Web service or an API server. In contract, a reverse proxy is the opposite of what a forward proxy does. It is a type of proxy server that retrieves resources on behalf of a client from servers.
Further, this post focuses on a reverse HTTP proxy over WebSocket, in brief, it uses the WebSocket protocol as a "tunnel" to pass TCP communication from server to client.
A case study in Go
There are not that many examples of implementation reverse proxy over WebSocket. In JavaScript, mhzed/wstunnel is well known, in Haskell, erebe/wstunnel is.
In Go, inconshreveable/ngrok and coyove/goflyway is well known, especially ngrok is popular among developers as a SaaS service.
In this post, we will focus on understanding the basic concepts and read prototypical and simpler one root-gg/wsp. wsp is developed by root-gg, which is a reverse HTTP proxy over WebSocket, whose aim is to securely make call to internal APIs from outside. It is difficult to use in production, but it is a good learning material to explain the design of reverse proxy over WebSocket.
However, maintenance has stopped since the days of Go 1.6, so I'll proceed with this post based on code hgsgtk/wsp that I forked and modified for the Go situation in 2021 (Thank you root-gg).
Demonstration of wsp
It's easier to get a general idea of how it works if you see it in action, so here's a brief demo. In the demonstration, the following four system component are introduced.
- Internal API(http://localhost:8081): the server you want to request via the reverse proxy
- wsp client: running in a network with the internal API
- wsp server(http://localhost:8080): running outside the network
- App: starting point for communication to the internal API
Try to send a request to the internal API via WebSocket communication to the endpoint /hello.
$ curl -H 'X-PROXY-DESTINATION: http://localhost:8081/hello' http://127.0.0.1:8080/request
HTTP communication was relayed by the following route.
app -> wsp server -(WebSocket)-> wsp client -> internal API
Here is a terminal image when sending a HTTP request from app.
The internal network restricts incoming requests from outside or does not have any global IPs, so that it is not possible for external server to connect to a client started in an internal network. For example, if the client is launched on your local PC, it is not possible to send a request to the client from an external server.
Therefore, the starting point is wsp client in the internal network. In this design, it starts with a request for a WebSocket connection from wsp client to wsp server.
The following points to implement this design will be explained in this post.
- Start a WebSocket server
I will explain the rest points in part 2 and beyond.
- Establish a WebSocket connection
- Keep a established connection
- Relay TCP connection from "App" to the peer of WebSocket
- Relay TCP connection in WebSocket data to "internal API"
Start a WebSocket server
The first step is to start up a WebSocket server which waits for requests from WebSocket clients.
As a side note, WebSocket is a mechanism for low-cost, full-duplex communication on Web, which protocol was standardized as RFC 6455. The following diagram, quoted by Wikipedia, describe a communication using WebSocket between client and server.
In the case of wsp, it starts like this.
$ ./wsp_server -config examples/wsp_server.cfg
{
"Host": "127.0.0.1",
"Port": 8080,
"Timeout": 1000,
"IdleTimeout": 60000,
"Whitelist": [],
"Blacklist": [],
"SecretKey": ""
}
Let's read the Go code. First is the main function (cmd/wsp_server/main.go), which is the entry point.
package main
import (
// (omit)
)
func main() {
// (omit)
server := server.NewServer(config)
server.Start()
// (omit: shutdown)
}
server.NewServer function returns a pointer of Server struct which represents a reverse HTTP proxy over WebSocket.
type Server struct {
Config *Config
upgrader websocket.Upgrader
pools []*Pool
lock sync.RWMutex
done chan struct{}
dispatcher chan *ConnectionRequest
server *http.Server
}
An important field is upgrader whose type is websocket.Upgrader to specify parameters for upgrading an HTTP connection to a WebSocket connection. In Go project, gorilla/websocket is widely used to implement WebSocket, and many samples using this library are available on the internet.
WebSocket is designed to work over HTTP. To achieve compatibility with HTTP, the WebSocket handshake uses the HTTP Upgrade header in order to change from the HTTP protocol to the WebSocket protocol.
Fields of Upgrader struct is as follow:
type Upgrader struct {
HandshakeTimeout time.Duration
ReadBufferSize, WriteBufferSize int
Subprotocols []string
Error func(w http.ResponseWriter, r *http.Request, status int, reason error)
CheckOrigin func(r *http.Request) bool
EnableCompression bool
}
If you want to know more about the WebSocket parameters or gorilla/websocket internal implementation, refer to this post Learn WebSocket handshake protocol with gorilla/websocket server.
In NewServer function, it creates a value whose type is websocket.Upgrader with default parameters.
func NewServer(config *Config) (server *Server) {
// (omit)
server.upgrader = websocket.Upgrader{}
// (omit)
return
}
Next, call server.Start function to start a WebSocket server.
func (server *Server) Start() {
// (omit)
r := http.NewServeMux()
r.HandleFunc("/request", server.request)
// (omit)
server.server = &http.Server{
Addr: server.Config.GetAddr(),
Handler: r,
}
go func() { log.Fatal(server.server.ListenAndServe()) }()
In this code, an HTTP server that exposes an endpoint /request is set up at a given address. The handler mapped to /request is as follow (code):
func (server *Server) register(w http.ResponseWriter, r *http.Request) {
secretKey := r.Header.Get("X-SECRET-KEY")
if secretKey != server.Config.SecretKey {
wsp.ProxyErrorf(w, "Invalid X-SECRET-KEY")
return
}
ws, err := server.upgrader.Upgrade(w, r, nil)
if err != nil {
wsp.ProxyErrorf(w, "HTTP upgrade error : %v", err)
return
}
// (omit)
}
There are several knowledges to implement WebSocket server, so I'll explain it line by line.
Authentication of WebSocket
The first line is:
secretKey := r.Header.Get("X-SECRET-KEY")
if secretKey != server.Config.SecretKey {
wsp.ProxyErrorf(w, "Invalid X-SECRET-KEY")
return
}
The WebSocket protocol does not handle authorization or authentication. However, it is a problem that anyone can access the WebSocket server without authentication just because the protocol is not prepared.
Business requirements often require authenticating the connecting WebSocket client and associating it with a registered customer record.
There are several patterns to implement authentication, and Heroku Dev Center - WebSocket Security describes one pattern - "ticket" -based authentication system.
- The client-side contacts the server-side to obtain an authorization "ticket".
- The server stores the ticket and returns it to the client
- The client opens the WebSocket connection, and sends along this "ticket" as part of an initial handshake.
- The server can compare this ticket, check an user's condition (i.e. source IPs)
See Authorization and IPs chapter in Armin Ronacher's post Websockets 101 for detail.
There are several options for where and how to tell the "ticket" from the client to the server. A commonly used method for simple authentication is to set the relevant ticket in the HTTP header. For example, Amazon API Gateway provides the option to require API clients to pass the API key as the X-API-Key header.
The sample code implements a similar concept. It validates the value in the header X-SECRET-KEY at initial handshake.
secretKey := r.Header.Get("X-SECRET-KEY")
Also, depending on your business requirements, there are several options for how to issue the "ticket". A idea is to issue "ticket" (or access key, or access token) in advance on their administration UI. Or if the process of asking where the WebSocket server to connect contains before handshake, it is also good idea to issue a one-time "ticket" by "access token" which is issued in advance.
In "ticket" -based authentication system, the server authenticates only at initial handshake, but there is also an idea to authenticate every message. See Klee Thomas's post Authenticating Websockets more detail.
Handshake (HTTP Upgrade)
After authentication, upgrade an HTTP connection to an WebSocket connection.
ws, err := server.upgrader.Upgrade(w, r, nil)
if err != nil {
wsp.ProxyErrorf(w, "HTTP upgrade error : %v", err)
return
}
websocket.Upgrader.Upgrade function does it. To put it simply, it checks if the handshake request from the client is valid and returns a successful handshake response if it is.
i.e. a valid handshake request
GET /register HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Origin: http://example.com
Sec-WebSocket-Version: 13
i.e. a successful handshake response
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
If you want to know the internal implementation of Upgrader.Upgrade function, see Learn WebSocket handshake protocol with gorilla/websocket server.
Conclusion
This post explained how to start a WebSocket server and WebSocket specification and security considerations to establish a WebSocket connection.
I will explain the rest points in part 2 and beyond.
- Establish a WebSocket connection (Part 2)
- Keep a established connection
- Relay TCP connection from "App" to the peer of WebSocket
- Relay TCP connection in WebSocket data to "internal API"
Top comments (3)
Hi Kazuki,
this is a really nice post about implementing reverse proxy!! :thumbs-up:
May I just ask you to increase the size of the "Here is a terminal image when sending a HTTP request from app." to make it more readable, please?
I got it! Thanks for reading it!
I’m going to fix the image tomorrow✍️
I's done!