Business is picking up. You're seeing more traffic to your web app and you know the next big traffic spike could put it over the edge. When that happens, your customers won't be able to do their job and it will cost them and your company money.
Today, you have three Amazon EC2 instances (dev, test, and production) that you set up manually through the AWS Console. You could probably recreate an instance again if you had to, but the last time you had to do that, you forgot to open up traffic to the Amazon RDS database and your customers experienced some downtime while you and the rest of your development team diagnosed the issue. Now that you need to handle more traffic, creating new instances by hand will not scale.
The dev and test environments are supposed to be identical to production, but let's face it — over the last year, they've gotten out of sync, occasionally causing application deployment issues in one environment or the other. And when these issues do happen, you have no idea who did it, when it happened, or the context of the change.
Now that all your infrastructure resources live in the AWS cloud, it would be even more time and cost-efficient for your development team to be able to quickly build their own dev environment in the cloud, do their own dev and testing, and tear it down when they're finished. This dev environment would have parity with the test and prod environments.
What if you could avoid all these challenges, scale your environment automatically, maintain environment parity, and even be able to audit changes to it? You can do all of this and more with Infrastructure as Code (IaC). Infrastructure as Code is the management of your infrastructure resources and their dependencies with code. In this post, we'll look at the benefits of using IaC over the manual and error-prone methods of the past, how it works, and some of the IaC framework choices you have.
How It Works
An Infrastructure as Code framework supports a high-level descriptive language where you describe your desired resource state and dependencies. When you execute this code, it interfaces with your cloud provider's APIs, figures out the differences between your desired state (in your code) and the current state (in the cloud). It will create, update, or delete resources when executed.
You are able to treat your infrastructure with the same quality and rigor as you do your application code. Your Infrastructure code is version-controlled, just like your application code. Cloud resources aren't changed manually anymore because your code is now the single source of truth. And if they are changed manually, something called drift, you have mechanisms to roll back those changes or notify your team.
Your code can be written with an iterative approach, just like you would your application code. Write the minimum amount of code that will work, test it, deploy it to the cloud, fix it, enhance it, extend it, write more code, and repeat the process. You'll find bugs earlier in the process and build only what you need from the start.
Why Codify Your Infrastructure
There are four main reasons why you would manage your cloud resources with code.
Scale Quicker and Easier
With the move to the cloud, our resources are more short-lived. They need to scale up and down and globally at a moment's notice. They do this more frequently and quickly than in the past. With IaC, we can automate the provisioning process to facilitate both scaling and quicker, repeatable deployments.
With the old process, if your team needed new infrastructure provisioned for a project, you'd have to submit a ticket and wait in line. You'd wait for other infrastructure projects ahead of yours as well as for the new physical infrastructure to be purchased, installed in a data center, and manually set up according to the project requirements.
Know Your Infrastructure Better
Now that code is the single source of truth for your infrastructure, it becomes self-documenting. You and your team can look at the code and immediately know the intended architecture of your systems.
Because your infrastructure is now defined in one place, code, you can version control it like you do your application code and take advantage of the same benefits. Benefits like versioning and change history, visibility into who made a change when, why, and even links back to your project management tracking system.
Control Your Risks and Costs
With infrastructure automation, your developers don't have to wait in line for a long infrastructure project to be implemented, and scaling and growing globally can be done much more efficiently. Because of this, you're able to go to market much quicker and make your software available to your customers when they need it, not when you can manually provision it.
With automation, you are less prone to human setup errors and application bugs related to environment inconsistencies. When these bugs are prevented from making it all the way through to production and to your customers, you reduce your risks and long-term costs.
Most defects end up costing more than it would have cost to prevent them. Defects are expensive when they occur, both the direct costs of fixing the defects and the indirect costs because of damaged relationships, lost business, and lost development time. — Kent Beck, Extreme Programming Explained
Improve Your Quality
When you automate your infrastructure, you get environment parity, where your dev, test, and production environments are identical. You get consistency between infrastructure deployments, where each resource is provisioned exactly the same way as others, exactly as you've defined it.
Your development and operations teams can be more aligned, using the same code, that single source of truth. Developers can provision their own environments that reflect test and production environments.
As with your software architecture, patterns will also emerge from your infrastructure. Operations teams can define reusable infrastructure components that developers can take advantage of. They can define how infrastructure must look under certain circumstances. For instance, maybe all your EC2 instances must have AWS Systems Manager Session Manager enabled.
Infrastructure as code also plays a key role in continuous integration and continuous delivery/deployment (CI/CD), which verifies your application throughout the Software Development Lifecycle from development and test through to deployment to production. It relies on infrastructure automation to create consistent environments for you to be confident that what you built works in your test environment and in your production environment for your customers.
You can also put your infrastructure code through the same CI/CD pipeline to iteratively develop, run automated tests, and deploy your infrastructure.
Framework Choices
There are a number of different Infrastructure as Code frameworks to choose from. Depending on your team's roles and skillsets, one framework might make more sense than the other. We won't dive into them all here, but in the next few posts, I'll cover use cases for AWS CloudFormation, the AWS CDK, and Hashicorp's Terraform.
Wrapping Up
Now that we know what Infrastructure as Code is and the benefits it gives us, let's look back at the problem you originally set out to solve and how IaC fits in. With IaC, you'll now be able to define your infrastructure resources and their dependencies in code, version control that code as your single source of truth, and automate the provisioning of your dev, test, and production environments. When it comes time to scale, you (or one of your teammates, since now that all that knowledge is out of your head, you've gone on vacation!) can modify the code that defines your infrastructure and re-provision it to handle your anticipated bump in traffic.
With this new process in place for provisioning new infrastructure, your development teams are not only able to deliver their work quicker but they are also able to try out new ideas and innovate more.
Ready to dive right into Infrastructure as Code? Get started with AWS CloudFormation with these five resources!
Like what you read? Follow me here on Dev.to or on Twitter to stay updated!
Top comments (4)
Great article! How’s about testing? What do you recommend for building testing and passing in isolation before deploying infra changes to prod?
Thanks Lee!
Excellent question. I've been digging into Chef InSpec, which allows you to write and run tests using a DSL on remote systems like servers, containers, and cloud resources. It can be used on any machine or resource regardless of whether it's managed by Chef, AWS CloudFormation, AWS CDK, Terraform, etc. or even manually by a human. Then, you can integrate these tests into your build pipeline, to verify your infrastructure in each environment and deploy, just like you would your application code. There are some other tools out there like Terratest.
Thanks for the reply 😎 Terratest looks interesting. Do you find that that IaC test and validation isn’t usually as widely adopted as app level TDD?
I don't have a sense of that quite yet but I think it will absolutely depend on your needs, just like where and when you integrate in automated testing at the application level.