Nginx as a Reverse Proxy

Nginx is a versatile tool: webserver, load balancer, reverse proxy. In this article, I show how to use Nginx as the central reverse proxy in your cloud that works with Consul and local DNS servers for providing well-known domain names of applications running in your cloud.

You know, sometimes a tool is great, but you wish to have more control, and you wish to truly understand the problems that a tool solves. This is my motivation for ... replacing the edge Router Traefik with Nginx¹.

Nginx is first and foremost a proven, resource effective, open source webserver. It also is, as of a march 2020 report, the number one webserver with 37% market share. Nginx is the web server that we employ at work to load balance services, to provide TLS, and as an ingress server in our Kubernetes cluster.

All good reasons to use Nginx in my infrastructure at home project as well. Nginx will fulfill several roles. First of all, it will be an edge router, connection external networks with the services that run in my cloud. Second, it will provide TLS encryption for client to service and for service to service communication.

In this article, I will show how to start and configure Nginx to work as a reverse proxy server.

This article appeared originally at my blog.

Custom Nginx Docker Image

The custom Docker image will be based on the official Nginx image. I will add two volumes: One for providing the config files, and one for providing static content. Here is the Dockerfile:

FROM nginx:1.17.9-alpine

VOLUME [ "/etc/nginx/conf.d" ]
VOLUME [ "/data/www" ]

We build the docker container.

>> docker build . -t inginx

Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM nginx:1.17.9-alpine
 ---> 377c0837328f
Step 2/3 : VOLUME [ "/etc/nginx/conf.d" ]
 ---> Using cache
 ---> b1fca48d47f8
Step 3/3 : VOLUME [ "/data/www" ]
 ---> Using cache
 ---> 26e85fbde6b0
Successfully built 26e85fbde6b0
Successfully tagged inginx:latest

Now we need to provide the basic configuration file /etc/nginx/conf.d/serve_static.conf. The file will create a Nginx process that listens on port 80 and it will serve the route /static by showing a directory listing of /data/www.

server {
  listen 80;
  location /static {
    root /data/www/;
    autoindex on;
  }
}

Then, provide some files for /data/www/ and run the container.

docker run -p 80:80 \
  -v $PWD/nginx.conf/:/etc/nginx.conf \
  -v $PWD/data:/data/www \
  --name inginx inginx:latest

Enter http://localhost:80/static in your browser, and you will see something like this:

Good. Now let’s evolve this basic config to a reverse proxy.

Nginx Reverse Proxy Configuration

In a nutshell, a reverse proxy is a server that receives incoming requests and forwards them to another server. In a production setup, you would also configure features like load balancing requests between different servers, applying modifications the request URI and modifying the headers. The example in our case will be to accept requests for the host grafana.infra and forward them to the same named service that is registered with Consul².

Let’s build the configuration file gateway.conf bit by bit.

DNS Resolution for Services

First, we need to understand the following Nginx configuration directives:

• resolver: The server that provides DNS resolution
• upstream: Defines one or a group of servers to which the traffic will be forwarded. Each server can be defined as an IP address or with a domain name.

The definition of an upstream server for the Grafana is this:

http {
  resolver 192.168.2.201 valid=10s;

  upstream grafana {
    server grafana.service.consul:9090;
  }  
}

As you see, I'm using the local DNS resolver to resolve the domain name grafana.service.consul - behind the scenes, Consul will provide the real IP for this service.

Forward Incoming Requests

Now we need to define how to accept and process requests. The relevant Nginx directives are:

• server: Defines a request handler
• server_name: This directive defines for which domain name the request handler is responsible, you can use multiple servers or wildcards in the domain name
• listen: The ports on which Nginx listens
• location: Each location block defines what to do when the URL matches a specific route
• proxy_pass: The server to which the incoming requests will be forwarded to

We want requests from grafana.infra to be passed to our upstream grafana server. The configuration is the following snippet:

server {
  server_name grafana.infra;
  listen 80;

  location / {
    proxy_pass http://grafana;
  }
}

This configuration works ... not completely. For some reasons, JavaScript resources cannot be properly loaded. When I run the Docker container, and then open browser console, I see this:

Why is that? The Grafana server does not know that a proxy was involved. We need to add additional HTTP headers for forwarding requests from Nginx to Grafana. These headers are:

proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

Their meaning is:

• proxy_redirect - Do not bounce traffic back to the receiving end
• Host - Keep the same host name from the original request
• X-Real-IP: Keep the original requests IP address
• X-Forwarded-For: A list of IP addresses telling the service how this request was routed
• X-Forwarded-Proto: Keep the requests scheme

Final Configuration File

Let’s put all of this together. The final configuration file gateway.conf is this:

http {
  resolver 127.0.0.1 valid=10s;

  upstream grafana {
    server grafana.service.consul:9090;
  }  
}

 server {
  server_name grafana.infra;
  listen 80;

  proxy_redirect off;
  proxy_set_header Host $http_host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;

  location / {
    proxy_pass http://grafana;
  }
}

Running the Nginx Reverse Proxy

Now we can run the Docker container, and access it in our browser.

docker run --network host \
  -v $PWD/nginx.conf/:/etc/nginx.conf \
  -v $PWD/gateway.conf/:/etc/gateway.conf
  --name inginx inginx:latest

The Grafana dashboard is properly rendered.

Conclusion

This article showed how to configure a Nginx Reverse Proxy for accessing services that are registered with Consul. The configuration consists of two blocks. The first block uses local DNS resolution to get real IP addresses for services registered with consul. The second block provides important HTTP header when forwarding incoming request to the Consul service. With this, you can define convenient domain names to access all Consul services.

In the next article, I will show how to add TLS encryption to Nginx.

Footnotes

---

1. And another reason is that Traefik was not verbose in access and error logs, I was trying to get it working but the system did not help me. Also, the official Traefik documentation seems to be written for very experienced admins, it lacks context and troubleshooting information. ↩

2. In case you did not follow the complete series: Consul is a service registry and discovery software that provides a DNS interface to find the dynamic IP and port of other services. I use this to register services, such as Prometheus, which are running as Docker containers. ↩