If you are beginning your journey in learning a cloud provider like Amazon Web Services or Google Cloud Platform you have likely come across the term Infrastructure As Code (IaC). It is a swiss army knife for both developers and system administrators. In this post, we are going to dive into what IaC is, the benefits it provides, and the different forms it can take.
Explain Infrastructure as Code like I'm Five
To put it simply, it is the practice of representing our infrastructure in some form of code. The definition of "code" is kept very loose in IaC as it could be YAML or JSON config files, homegrown languages like HCL, or traditional languages like NodeJs or Ruby.
While IaC is often talked about within the context of the cloud it's not limited to just our cloud resources.
As we know there are all types of infrastructure. There are public cloud and private cloud resources. Cloud and on-premise resources. Or a combination of all of these sometimes referred to as hybrid cloud.
Therefore at the highest level, we can summarize IaC as the process or practice of representing the infrastructure our systems run on in a common language. This language makes the infrastructure visible to the entire team and is reproducible with relative ease.
The Benefits
OK, we get it, infrastructure as code is...just code.
But why in the world should we care to use it and what are we going to get out of it?
In my opinion, you don't see the benefits until you have experienced provisioning infrastructure by hand. Have you ever followed a tutorial where you clicked this button, that button, this other button, and a few more buttons until you had something set up with your cloud provider?
The answer is most likely yes.
Maybe you did that for your test environment. Then when it came time to go to production, you had to click those same buttons again and hope you get the order right.
What if someone else on your team had to reproduce the same button clicks in order to create their own environment? See how this is getting painful. Some folks might say, document the process and that solves the problem. This is true but it's still not ideal.
Documentation shares knowledge initially but can become stale. Developers have a nasty habit of not reading the documentation if it isn't right in front of them. Lastly, documentation can't be tested in the same way code can.
This is the biggest benefit infrastructure as code provides, automated, reproducible, testable, and self-documenting infrastructure.
By representing resources as code, we can parameterize the code to support multiple environments, share the code with our teammates, and even test the code to ensure accuracy.
Just like changes in our application code are tested, peer-reviewed and consumable by everyone on the team. The process of provisioning, maintaining, and updating our infrastructure can slip right into the same flow using IaC.
Potential downsides
While I believe the benefits of representing your infrastructure as code far outweigh the downsides, it is still worth knowing the traps we can fall into.
The initial downside of infrastructure as code is the learning curve. Like I mentioned earlier, the term code is flexible in this process and therefore it often requires learning a new flavor of code based on the tool you are using. Learning how to represent your VPC networking stack inside of AWS CloudFormation is a non-trivial skill to learn.
Piggybacking on the learning curve, time is another potential downside of IaC. It is often a lot quicker to go into your AWS Console and click a few buttons than it is to write infrastructure code inside of Terraform. Many teams make this tradeoff early on for the sake of going faster and shipping a product or service.
The final trap is infrastructure divergence. Once we commit to the path of infrastructure as code we must stick to our guns. Changing an infrastructure stack by hand in the console that has been provisioned by an IaC tool can cause divergence. Meaning the state of our infrastructure has changed because it was edited manually and our IaC tool doesn't know that.
While these are significant downsides they shouldn't stop us from exploring IaC further. However, they should be things we consider as we are choosing tools that help us on this journey.
Tools, Tools, Tools and more Tools
When it comes to IaC tools it is helpful to think of them in two different, but often overlapping categories.
- Configuration orchestration tools are designed to automate the deployment of our infrastructure.
- Configuration management tools are designed to help configure the software and systems on infrastructure that has been provisioned.
It is important to note that these categories often overlap. As we will see some IaC tools allow you to orchestrate/provision infrastructure as well as configure it. Additionally, as infrastructure gets more complex it is not uncommon to see both types of tools used.
With me so far? Excellent. Here are some of the current tools that are popular in the infrastructure as code space.
Terraform is a cloud agnostic infrastructure provisioning tool. It allows us as developers to represent our infrastructure, regardless of whether it is AWS, Google Cloud or Azure, in a well-defined language known as HCL.
Terraform is highly extensible via plugins and has a very strong community around it that produces great open source modules for provisioning chunks of infrastructure.
AWS CloudFormation is not cloud agnostic and is dedicated to provisioning infrastructure within AWS accounts. It allows us to represent our AWS infrastructure in JSON or YAML configuration files that we can then execute to create, update, and destroy resources.
CloudFormation is very well integrated into the AWS ecosystem and has developed a lot of features for previewing, rolling back, and managing resource changes.
Chef falls under the configuration management tooling. It allows us to create recipes and cookbooks that define the exact steps needed to reach the necessary configuration of our application.
Chef, like Terraform, is cloud-agnostic. It uses the commonly known language, Ruby. It is typically used for configuring Elastic Compute (EC2) instances and even on-premise servers.
Puppet is another Ruby-based configuration management tool similar to Chef. The difference between the two is that Puppet is known as a declarative tool. This means that we as developers define what our infrastructure is supposed to be and Puppet figures out how to make that true.
Ansible is an infrastructure automation tool from Red Hat. Developers describe how their components and system relate to one another. It is meant to manage and codify systems end to end rather than independently. These definitions are written in YAML and are known as Playbooks.
Juju is an IaC tool from Ubuntu which allows developers to represent their infrastructure as charms which are sets of scripts that deploy and operate systems. These charms can be packed together as bundles to deploy the entire infrastructure for an application.
This is definitely not a comprehensive list. There are constantly new tools being developed in both the configuration orchestration and management space. But for right now these are the mainstream tools that are being used across the industry.
Where to go from here
We now know the definition of infrastructure as code, the benefits it provides as well as the pitfalls, and we now know a few tools to explore that allow us to provision our resources via code.
The next step in understanding IaC is to pick a tool from the list above and begin exploring it.
- Start with a simple use case like provisioning an S3 bucket or EC2 instance.
- Once the resource is provisioned. Explore another resource to provision.
- Experiment with parameterizing the code to support multiple environments.
- Learn the differences between creating, updating, and destroying resources for the tool.
Once we have explored a tool that enables us to represent our infrastructure as code we can then explore the differences between that tool and others in the list above.
Conclusion
Infrastructure as code is the process and practice of representing underlying resources that support our systems in code. This allows us to use the same processes we use already when writing application code.
This includes sharing the infrastructure with our team so that the team can maintain the resources collaboratively. It also includes testing how our infrastructure is created, updated, and deleted. Just as with application code the definition of our infrastructure can be peer-reviewed.
Infrastructure as code is a swiss army knife in the toolbelt of DevOps. It is one that can be leveraged by teams big and small regardless of whether that cloud is provisioning resources in AWS, GCP, Azure, or their own data center.
Are you hungry to learn more about Amazon Web Services?
Want to learn more about AWS? I recently released an e-book and video course that cuts through the sea of information. It focuses on hosting, securing, and deploying static websites on AWS. The goal is to learn services related to this problem as you are using them. If you have been wanting to learn AWS, but you’re not sure where to start, then check out my course.
Top comments (16)
Pulumi looks awesome. I'm really new at managing any type of modern infrastructure but the abstractions that Pulumi offers are really amazing. I hope it grows quick and they refine some rough edges because being able to code your infrastructure is pretty cool :)
Pulumi is one that I meant to include. It looks very promising as it allows you to represent your infrastructure right alongside your application code.
Exactly. He best thing is that by being able to write just code it allows many patterns that are quite cool.
If anybody is interested in the views from a newcomer in the infrastructure world I wrote about it in my blog.. I linked there to an interesting use case, AWS Lambda Warmer as Pulumi Component.
Good notes on this! We've been getting more and more into developing our resources this way, but it still doesn't mean you get away from the Administrative headaches of having to deal with AWS console. Especially when trying to configure your VPC's, IAM Users and Roles just so you can get them into your Terraform templates. I don't mind learning new tools, after all you need to keep your toolbox current, it just gets so overwhelming when even in my environment we are using Ansible, Terraform, Docker, and probably one or two more I am forgetting just to get your environments in place. At times its hard to keep up.
Michael, I think you hit on a major point that is evolving in this space. Often times more than one tool is being used.
For example, I can use Serverless Framework to provision my serverless resources and Terraform for my VPC, roles, and deployment pipeline. I could also have a Terraform template that provisions EC2 instances in AWS, and maybe I even use Puppet to configure things on those machines.
In that basic example, I am already using three tools.
I don't have the answer here, but I think the future is a bit unknown in terms of this level of complexity. Is this the right way? The wrong way? Unclear.
The one thing that seems clear to me is that an abundance of infrastructure as code is better than no IaC.
I've been in this space before and in my experience all the tools end up finding either their niche or evolve so that they become the "standards" for specific groups. What we had leading up to this was a lot of flavor of the month that people in those positions decided was the one we wanted to use NOW.
Though I started with a bit of venting on the complexity its something that's endemic in our space, we are constantly moving forward. Something new comes, we take it and check it out, if we love it add it to our toolbox or drop it off. The next person who comes in either learns from what we did, if we mentor, or makes their own decisions if the company has not set a standard.
It's nice to keep learning, its the one thing that excites me about going into work every day - the intellectual challenge - but it can be draining at times.
The problem is getting more apparent every year.
We're introducing more and more abstraction layers on top of physical infrastructure, making compromises, generalizing and reducing use cases to most common ones creating bottlenecks and slowdowns ignoring special requirements.
Instead of physical infrastructure we have hypervisors, running virtualized OS-es with virtualized hardware that run containers that run frameworks that transpile to intermediate languages that are translated to binary code that gets executed.
Instead of switches and routers we have SDNs that execute on CPUs sharing execution with all cloud customers. No wonder they do not ofer any SLA on the bandwidth, it's impossible to keep it!
You want cloud? For the price you get average performance shared with tons of other cloud customers, no options to resolve resource contention or hardware overutilization or any other random slowdown you experience.
They claim power efficiency, performance and internet speed has increased several ten folds over past years. I hardly noticed webpages or mobile apps loading any faster.
Are you wondering why you need new computer each time new version of Windows comes out? Shouldn't they finaly manage to fix all performance issues and make everything run smooth? Especially when SSDs now reach 5x the speed of traditional HDDs? When we have 8-16x more memory than we used to have? When we have 4-8x more CPU cores than before?
Where does all the power go?
Whoa there Slavius, what are you so mad about? Your your profile says you are " Currently working as a freelancer on infrastructure, application architecture and cloud engineering."; but then you dig into them like it's going out of style. What's up? It's like you know the differences, but run away from the reasons.
I'm not sure what upset you so much from an IAC article to begin lamenting and ranting about all of this with your ill-informed opinion. Please have yourself a better day/week/year fam.
I'm unfortunately not interested to reply to your comment for obvious reasons - you are part of a cult and I don't buy it.
You are the one ill-informed. I am long term cloud end-user, renting physical servers and several years working as part of cloud provider company. I have my reasons to write what I write and those are facts I face every day.
Good day, sir!
Not everything you said is truth. What you described as a cloud is commonly known as public cloud and you share resources with other tenants.
But then there's private cloud where tenant will be given its own resources that won't be shared with anyone else.
Also, public cloud is far more performant for cloud provider because they can use hardware more efficiently by not having it idle.
You get your resources that may not be shared with anyone else - while at the same time they are far from being neither performant nor flexible.
You get the same cheap CPUs bought for public cloud because their ROI is so fast (lot of core count, small caches, low frequencies) you get one fixed storage of unknown configuration that will 100% not suit your use case. You will be lucky if it's SAS based, in most cases it's NLSAS OR staight SATA. You luckily may get your own RAID controller with some cache but no control over read/write ratio. In most cases tho, you get shared NAS with other tennants because that's where they invested tons of money and they need to monetize on it.
Then you get a network connectivity. That's also shared. Not only with tennants but while doing your backups it will cut from it. If you're lucky they did invest into dedicated NIC for NAS connectivity.
How is that better than public cloud? It is simply not!
Folks are gonna like this one
one thing to keep in mind, that terraform, comparing to ansible, offers more declarative way of describing desired infrastructure - if in ansible you say “add one node”, in terraform you say “I want to have 10 nodes”
both approaches have their pros and cons )
Really enjoyed this read Kyle. Loved how you broke things down and did a great job of weighing both the pros and cons of Infrastructure as Code (IaC). Need to look into learning more about this. Also thanks for spelling out some of these acronyms, learned a few new ones :)
Thank you for the comment Nick.
One thing I don't touch on here is the idea of moving from manual provisioning to IaC and converting existing infrastructure. This is a tricky problem and requires even more time and effort because we don't want to break production.
Therefore, my advice is to start with IaC on any new services/resources. Establish the culture and process around it. Get used to operating in this way and then tackle your existing infrastructure.
Great post. I hope to at some point be able to finally get the time to finally implement this at my work. We really need it.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.