There are multiple, let’s say, philosophies when it comes to the cloud costs.
Within bigger enterprises, this philosophy can often be described as “we don’t care”.
Similar philosophy is often applied by well-funded startups - till they start running out of money, of course.
Bootstrapped companies, on the other side, are normally more conscious about spending money, even though, sometimes, way too conscious.
At mkdev, we believe that cloud architecture and cloud costs go hand in hand, you can’t separate one from the other when you run your application in a cloud native way. And we can understand a lot about your architecture and infrastructure just by looking at your cloud bill, during our cloud audit service, for example.
And it makes all the sense to make cloud costs, same as the cloud infrastructure, part of your automation, to make costs a vital part of your observability setup, as well as CI/CD pipeline.
In this article, we will examine one of the tools that makes this a bit easier, as well as discuss some shortcomings of such tools.
The tool is called InfraCost, and it’s a mix of an open source utilities and paid offerings.
At the core of the InfraCost lays Cloud Pricing API. The team behind the tool implemented a scheduled job, that fetches all the prices from AWS, Azure and GCP, and makes it available over a GraphQL-based API. You can even host this API inside your own network.
On top of this API go all the other components. One of them is an infracost CLI. You can use this CLI to get a breakdown of the costs for all the resources in your Terraform Code. Worth mentioning, that at the moment InfraCost works only with Terraform.
For example, I can get an information about all the costs of this internal mkdev EKS module. In a very nice output, InfraCost CLI shows us how much each particular resource will cost, and a total cost per month.
Using this CLI manually is fun, but not that useful in the long run. InfraCost developers realize that, and they offer an excellent collection of templates for some of the most popular CI/CD systems. You can just copy the GitHub Actions workflow they offer, configure an API Token to a GitHub Actions Secret and you get this beautiful cost estimates inside a Pull Request.
With cost estimations being part of your CI/CD pipeline, you are always aware of how the changes to your infrastructure impact your costs. The next logical step could be to use the cost policy feature, to prevent an expensive change from being merged.
Infracost is not the only tool that integrates cloud costs into your daily infrastructure as code work. Terraform Cloud, for example, also has this feature built-in, though only as an extra paid feature. env0 is another one - it has a lot of features, costs monitoring being only one of them.
Making cloud costs part of your automations is a great way to increase awareness about your cloud spending, but there is a catch.
With all of those tools, you will only get an information about resources that cost fixed amount of money per time unit.
But one of the main benefits of any cloud is the elasticity. Your infrastructure is supposed to grow and shrink depending on the load. With serverless applications, you pay only when your code is running. And with many, if not most, of the modern cloud services you are not paying based on time, but per usage. For example, you pay for the number of messages in the queue, or records in the stream, or a number of API calls to your S3 bucket.
This kind of costs can not be derived from your infrastructure code, and, while Infracost is planning to integrate your cloud-provider’s metrics to show these estimates, it still can’t look in the future, simply because it does not know your particular application and how it works and uses the cloud - and whether it does it correctly, of course.
Tools like Infracost can help a lot with bringing the costs to your infrastructure development workflows, but they don’t give you the full picture, neither they can help you to optimize your architecture to be more scalable, efficient and at the same time cost-effective.
If you are struggling with an ever-growing cloud bill, you will definitely benefit from our in-depth cloud audit. We go beyond the long-hanging fruits and take a real deep look at your infrastructure and applications, interview your teams and give you a detailed roadmap with all the possible improvements and changes we think you should do to improve scalability, security and stability of your cloud environment. And if you let us, we will even fill in your backlog with actionable tasks that you will be able to execute on your own.
Here's the same article in video form for your convenience:
Here's the original article by Kirill Shirinkin on mkdev.me
Top comments (0)