Enriching web requests using a code-first proxy

This post looks at using a reusable, configurable, code-first proxy to provide a consistent, controllable and maintainable way to enrich web requests.

The problem

If often find myself in a situation where I need to run some organisation specific, but otherwise generic code at, or around, the API entrypoint to a service. There are a plethora of options for where you could perform this task, but each comes with its own challenges:

Option	Distance from service	Challenge
Request Middleware	Same application	Tightly coupled to service - updates are HARD!
HTTP Filter	Same web server	Ties you to a web server - not really the done thing anymore
Sidecar	Same node	Large overhead to build and run
Service Mesh	Same node	Usually not great support for custom code
Reverse Proxy	Same logical system	Large overhead to build and run
Infra-first devices (Load balancer / API Gateway)	Same organisation	Usually not great support for custom code

A solution?

The solution that I will describe in this post could be used as either a sidecar or a reverse proxy, depending on your level of infrastructure abstraction.

The example system contains three components, the service, the proxy, and a database.

The code can be found here

I run all components locally using Docker for Mac. The Dockerfile for the proxy and the service are included in the repo. The containers are run on a user-defined bridge network, created by

docker network create mybridge

The proxy

For the proxy I envisioned a slim service that enabled a developer to slot in some custom functionality in the most straightforward manner possible.

Around this time Damian Hickey announced ProxyKit, which seemed to be exactly what I was after, so I set about creating a solution using it.

Damian Hickey ❄️

@randompunter

Looks like ProxyKit is a top trending C# project on github. It seems it fills a need.

#unashamed-self-promotion

14:48 PM - 06 Jan 2019

6 75

My proxy will look at the Authorization header for a user, look up information about the user in a datastore, and then add that information as a Header before forwarding the request to the downstream service.

This simple proxy is configurable in a number of ways -

• The 'alias' (ie. ip address, container name) and port of the downstream service
• The connection details for the database
• The information to retrieve from the database
• The amount of time to cache the data retrieved from the database

docker run -p 49161:80 -d \
--network=mybridge \
-e Downstream:Alias=api \
-e Downstream:Port=8080 \
-e Database:Host=postgres \
-e Database:Password=mysecretpassword \
-e Permission=HasSpecialPower \
-e CacheTimeSeconds=60 \
mattyjward/proxysidecar:blog

The database

For the database I used a fairly vanilla Postgres db, it just has a 'Users' table with some 'role' information.

docker run -p 5432:5432 -d -e POSTGRES_PASSWORD=mysecretpassword \
--network=mybridge \
--name=postgres \
--mount=type=volume,src=posgres,dst=/var/lib/postgresql/data \
postgres

The port is only exposed here so that I can connect directly to it and configure data - like so

psql postgres -h localhost -U postgres

and then

CREATE DATABASE Users;
\connect users
Create Table Users ( Name TEXT PRIMARY KEY, HasSpecialPower bool);

The service

For the service I created a simple Node app built on express. It looks for a header that the proxy sets, and uses it to make its authorization decision.

This does introduce some coupling between the proxy and the service.

docker run -d \
--network=mybridge \
--name=api \
mattyjward/permissionapi:blog

Run it

Once you have all three containers running, fire a request at your proxy server

curl -I localhost:49161 --header

Initially you should get a HTTP 401 Unauthorized response. This is being returned from the proxy service. When there is no Authorization header it will not even bother forwarding the request to the service.

So we'll add the header

curl -I localhost:49161 --header "Authorization:matt"

Now you should get a HTTP 403 Forbidden response. This is being returned by the service. The proxy could not find the user with the desired permission in the database, so did not add the header that the service is looking for.

Using psql, add a user to the database and give them the permission they need.

Insert into Users (Name, HasSpecialPower) Values ('matt', true);

Resend the previous curl http request (make sure you have waited for the cache to expire) and voila! you should get a HTTP 200. This is being returned by the service, after the proxy found the user with the permission in the database and added the header to the request.

What next

It would now be straightforward to spin up a second instance of the proxy which is appropriately configured for a different service.

This code is a fair way from production quality, but if I was to take it further I would look at

• replacing the Authorization header handling with proper OAuth JWT token validation
• using more complex authorisation policies
• using a container orchestrator

Thanks!