The Domain Name System or DNS is the main way traffic is routed on the internet. It can be used to route traffic to one or many servers and can be chained so that the same url can lead to different locations at different points in time. This is what enables blue-green deployment of software and releasing new versions of software without downtime.
So how can we use DNS to blue-green our software? In this article we will go through the process step by step.
What is Blue Green Deployment?
First we need to understand what blue-green means. Blue-green deployment is the process of provisioning more than one production environment so that you can set up a new version before cutting over to it.
By having both environments running you can quickly switch over to the new version which minimizes downtime and allows you to switch back to the previous version if needed. This is in contrast to an in-place update that relies on stopping the production environment, updating it, and starting it again.
More information about blue-green deployment can be found here in Martin Fowlers blog post from 2010.
1. Deploy A Version of Our Software with an A Record
In order to make use of blue-green deployment via DNS we first need to deploy a running version of our software. We can deploy a single host or multiple and create a DNS Record pointing their IP addresses. This will let us consistently access our service without needing to know the hosts IP addresses.
For this DNS record we should use an A record, which is a type of DNS record that maps hostnames to IP addresses. There are multiple types of DNS records with different purposes and A records are the most basic. We need one A record for each deployment.
The best way to do this is using a version number in the A record and deploying a new set of infrastructure for each version to achieve immutable infrastructure and rolling deployments. It is also possible to just have two static environments that could be labelled blue and green.
2. Create a CNAME DNS Record for our Application
The next DNS record we need to create is a CNAME. A CNAME is an alias from one hostname to another, this means it can be used for redirects or to provide multiple names for a single service. We should provision a CNAME that our customers use for accessing our service so we have control of the routing.
This record should not be versioned as there is only one per environment and we use it as a lookup for where traffic should be routed. Here are some examples of what the initial CNAME and A record might look like:
CNAME Record -> A Record logging.company.com -> logging-01.company.com logging.company.com -> 01.logging.company.com logging.company.com -> logging-blue.company.com logging.company.com -> blue.logging.company.com
3. Deploy a New Version of our Software with an A Record
After our system is running and customers are using the CNAME for accessing our service we can deploy a new version of the software. It can be deployed right next to our current version and once deployed we should be able to access it via its own A record.
Next we should run automated integration and regression tests so that we are confident the environment is working as expected and we can move onto the cut-over. It is important to make sure these tests don't impact the other deployments as they may share databases and other resources.
4. Update our Applications CNAME DNS Record
To cut-over our application to the new version we just update our CNAME record to point at the new A record. This should be quick but depending on the time to live (TTL) of the DNS record may take some time to propagate since DNS is cached in multiple places based on the TTL provided. I usually recommend 60 seconds as a good standard for DNS TTL.
If any errors occur or we aren't happy with the new environment we can revert by redirecting the CNAME back at the old A record. This should be quick and easy to do due to the simplicity of the process.
5. Cleanup
After the cut-over we may need to clean up the previous environment by deleting it or uninstalling any software that is left running. This can happen immediately or after some time but should be done as needed based on your specific environment.
Summary
Blue-green deployments are critical to minimizing risk and downtime of software releases and DNS is the main way to do so. There are many variations of this process but it is important to keep the process simple. Let me know if this is how you release new versions of your software and if there are any other parts of DNS releases you would like me to cover.
For more content like this follow or contact me:
- Twitter: @BenTorvo
- Email: ben@torvo.com.au
- Website: torvo.com.au
Top comments (3)
Are you concerned about having a lower TTL might increase the risk of DNS hijacking?
Generally no, you still have control of the DNS records so the risk should be similar regardless of the TTL?
Well you do only control the authority server at best, and at worst you control none of it but only trust your DNS records to be adhered to by a provider. Now that's very reasonable in the first place, and for a lot of people it's the way to go, but it does not prevent DNS hijacking or DNS poisoning. Neither does a high TTL, for that matter, but it is a preventative meassure that is possible to take to reduce the harm done if a DNS hijack were to occur.
With a low TTL during a DNS hijack, the DNS records will in the best case only be cached by clients for a very small duration of time. With a high TTL, the DNS records will in the best case be cached by clients for that much longer, making it possible that a DNS hijack never actually happens. Or only affects a minority of clients, perhaps. In the best case.
Another step that can be taken is using DNSSEC of course, which should help mitigate any issues with DNS hijacking, but there's no guarantee that DNSSEC will be used or even correctly implemented across a given network.
Anyway, it's just something to continually consider, and I won't pretend to have all (or even any) of the answers to these problems.