Necessary components of cloud VMs
david karapetyan Nov 04, 2017
When setting up a VM in the cloud there are some associated components that you should figure out ahead of time to make your life easier in the long run when managing those VMs. Here's a diagram of what I consider to be the bare minimum required components (I'm using AWS terminology but there are analogs for each component in other cloud providers)
I'll explain why each piece is important.
Presumably you have some software that is specific to your problem domain. Things like specific versions of Ruby and Python interpreters, database libraries and drivers, web servers like nginx, load balancers like HAProxy, key/value stores like Redis or Consul, etc. These things don't change often so for each type of VM that you will have in your fleet you should just bake the required software into the VM image ahead of time. In the past I've used Packer from HashiCorp but there are other tools as well so pick what you are comfortable with and use it to generate AMIs. There are many benefits to doing this, least of which is faster startup for the VM because you don't have to install things every time you provision a new VM.
Some things can't be baked into the image for one reason or another so for those things you should consider having an associated cloud-init script that will do the necessary dynamic configuration when the VM starts. This could be things like registering the VM in Consul, pulling in some static assets from S3, doing some basic health and sanity checks, downloading the configured version of your software, getting the required secret tokens from KMS (Key Management Service), etc.
Some people bake secret tokens into the AMI. This isn't necessarily good or bad but it means the AMI now has some long lived secret tokens and in general long lived secret tokens are a bad idea. You can avoid the problem of long lived tokens by associating an IAM profile to the VM and letting AWS handle ephemeral API tokens to interface with whatever other services the VM needs to interface with. I mentioned above you might need to talk to KMS to grab some secret tokens and using a IAM profile with properly restricted access rights is the right way to do that. IAM profiles also provide policy based restrictions on what things can and can not be accessed from the VM and even though the IAM policy language can be somewhat daunting it is worthwhile to invest some time in learning it because it is a necessary component of a secure cloud deployment.
Presumably your VM will be doing some work and maybe providing some network services accessible over network sockets. By associating a security group for the network access rules to each type of VM you will reduce a lot of suffering for yourself in the long run. I'm speaking mostly from experience and associating the security group with the VM instead of some other artifact leads to a much less complicated and more manageable network topology. Security groups also act as markers and the ID of the security group can be used in the ingress/egress rules of other security groups. For example, the load balancer running HAProxy should be able to access your application servers and your application servers should be able to access the database server but the load balancer should not be able to access the database. This is very easy to express if each type of VM has a unique security group associated with it and much harder to do if security groups are not associated with VM types and are associated with ports and protocols.
Each VM should come with a standard set of tags. I have my own personal preference for what this set should be but there is no right or wrong way to tag VMs as long as you're consistent about it. The tags I like are Name, Purpose, and Type. Name is obvious I hope. Purpose is some description for why the VM was provisioned. Type is used for grouping a class of VMs and makes it easier to query for specific types of VMs. For example, I'd use 'DB' for the database servers, 'LB' for the load balancing servers, and 'Application' for the application servers.
Questions and your own variations for what should be considered required components for deploying cloud VMs welcome.