Tracking AWS cost continues to be a very popular topic in the world of the cloud. The reasoning is often because folks are getting surprised by their bills at the end of the month. Or perhaps they thought a service was going to cost X, but at the end of the month it ended up costing 2X.
This isn't just a problem for those of us that are new to AWS. This is an issue that even the most seasoned AWS developers are running into.
Take me for an example, this morning I woke up to an AWS bill that was $20 more than what I typically pay each month. I have a billing alarm set up for my entire account, but the total bill still came in under that so it wasn't breached.
So I got to digging around and discovered that the additional spend was because I left a Load Balancer running in my account that I was using for testing.
This scenario is very common for folks because it is very easy to forget about a particular resource you created. Sometimes this is because we created the resource manually in the AWS Console. Or we created some infrastructure via code, but then neglected to tear it down when we were done with it.
For one of my products, the monthly cost for my RDS and EC2 spend is very stable and predictable. But in this case, because I left a Load Balancer running, my EC2 cost was $20 more than usual. So I set out to fix the issue of not knowing I left a Load Balancer running.
The goal of this post is to demonstrate how we can leverage AWS Budgets to create per service level billing alarms. We are going to use Terraform to create our budgets so make sure you have that installed and in your path.
What we want to track
Ultimate Fantasy Supercross is a fantasy motorsports platform I launched about three years ago. It uses a couple EC2 instances, an RDS database, and a static website in S3. The cost of my infrastructure is very predictable at this point.
| Service | Monthly Cost ($) |
|---|---|
| Elastic Compute Cloud (EC2) | 19.99 |
| Relational Database Service (RDS) | 35.04 |
| Simple Storage Service (S3) | 1.50 |
As mentioned earlier I already have an account level billing alarm configured that notifies me when my bill is approaching the $60 mark. But I also want to know when my forecasted spend is going to breach my monthly expectation for EC2 and RDS.
To accomplish this we are going to create two AWS Budgets, one for EC2 and one for RDS, to send me an email when my forecasted spend for each of these is going to exceed my expectation.
Creating our budgets
We are going to create three resources in our Terraform template. The first will be an account level billing alarm just in case you don't already have one. Then we will create two budget resources for our EC2 and RDS forecasted spend.
Let's begin with our account level billing alarm.
provider "aws" {
region = "us-east-1"
}
resource "aws_cloudwatch_metric_alarm" "account-billing-alarm" {
alarm_name = "account-billing-alarm"
comparison_operator = "GreaterThanOrEqualToThreshold"
evaluation_periods = "1"
metric_name = "EstimatedCharges"
namespace = "AWS/Billing"
period = "21600"
statistic = "Average"
threshold = "60"
alarm_description = "Billing alarm by account"
alarm_actions = ["<your-sns-topic-arn-for-notification>"]
dimensions {
Currency = "USD"
LinkedAccount = "<your-aws-account-id>"
}
}
Here we have our account level billing alarm. It is tracking the EstimatedCharges for an individual account. When that account reaches $60 USD in estimated charges it will send an alarm notification to the SNS topic we specify in alarm_actions. You should replace alarm_actions with the ARN of an SNS topic that has your email subscribed to it. You also need to replace LinkedAccount with your own AWS account id.
We can create our account level alarm by first initializing our template.
$ terraform init
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "aws" (1.39.0)...
Once our template is initialized we can apply our template to our AWS account.
$ terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ aws_cloudwatch_metric_alarm.account-billing-alarm
....
....
aws_cloudwatch_metric_alarm.account-billing-alarm: Creating...
actions_enabled: "" => "true"
alarm_actions.#: "" => "1"
alarm_actions.321893454: "" => "<your-sns-topic-arn-for-notification>"
alarm_description: "" => "Billing alarm by account"
alarm_name: "" => "account-billing-alarm"
arn: "" => "<computed>"
comparison_operator: "" => "GreaterThanOrEqualToThreshold"
dimensions.%: "" => "2"
dimensions.Currency: "" => "USD"
dimensions.LinkedAccount: "" => "<your-aws-account-id>"
evaluate_low_sample_count_percentiles: "" => "<computed>"
evaluation_periods: "" => "1"
metric_name: "" => "EstimatedCharges"
namespace: "" => "AWS/Billing"
period: "" => "21600"
statistic: "" => "Average"
threshold: "" => "60"
treat_missing_data: "" => "missing"
aws_cloudwatch_metric_alarm.account-billing-alarm: Creation complete after 1s (ID: account-billing-alarm)
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Great! We now have our account level billing alarm to notify us when our estimated bill is going to be greater than $60 USD. Let's create our budgets so we can alarm when our forecasted spend on EC2 or RDS exceeds our expectation.
Go ahead and add the following resources to your Terraform template.
resource "aws_budgets_budget" "ec2-forecast-alarm" {
name = "budget-ec2-monthly"
budget_type = "COST"
limit_amount = "20"
limit_unit = "USD"
time_period_start = "2018-01-01_00:00"
time_unit = "MONTHLY"
cost_filters {
service = "Amazon Elastic Compute Cloud - Compute"
}
}
resource "aws_budgets_budget" "rds-forecast-alarm" {
name = "budget-rds-monthly"
budget_type = "COST"
limit_amount = "36"
limit_unit = "USD"
time_period_start = "2018-01-01_00:00"
time_unit = "MONTHLY"
cost_filters {
service = "Amazon Relational Database Service"
}
}
Here we have our budgets for EC2 and RDS. Notice the cost_filters are using the fully qualified names for each service, this is what AWS budgets via Terraform expect. We can go ahead and run our apply command again to create our new budgets.
$ terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ aws_budgets_budget.ec2-forecast-alarm
id: <computed>
account_id: <computed>
budget_type: "COST"
cost_filters.%: "1"
cost_filters.Service: "ec2"
cost_types.#: <computed>
limit_amount: "20"
limit_unit: "USD"
name: "budget-ec2-monthly"
name_prefix: <computed>
time_period_end: "2087-06-15_00:00"
time_period_start: "2018-01-01_00:00"
time_unit: "MONTHLY"
+ aws_budgets_budget.rds-forecast-alarm
id: <computed>
account_id: <computed>
budget_type: "COST"
cost_filters.%: "1"
cost_filters.Service: "rds"
cost_types.#: <computed>
limit_amount: "36"
limit_unit: "USD"
name: "budget-rds-monthly"
name_prefix: <computed>
time_period_end: "2087-06-15_00:00"
time_period_start: "2018-01-01_00:00"
time_unit: "MONTHLY"
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Just like that we now have two budgets configured in our AWS account. The first tracks our spend on EC2 and the latter tracks our spend on RDS. However, we do not have notifications set up for these yet. This is an open issue in Terraform to add this support to the aws_budgets resource, but at this point, we have to do this ourselves.
Adding notifications to our budgets
Because of the limitation of Terraform as of today, we need to add the notifications to our budgets outside of our template. We can quickly add notifications to each of our budgets by using the AWS CLI. To get started you need your AWS account id once more.
Got it? Great, now we can run the following commands on our command line.
$ aws budgets create-notification --account-id <your-aws-account-id> --budget-name budget-rds-monthly --notification NotificationType=FORECASTED,ComparisonOperator=GREATER_THAN,Threshold=100,ThresholdType=PERCENTAGE --subscribers SubscriptionType=EMAIL,Address=<your-email-address>
$ aws budgets create-notification --account-id <your-aws-account-id> --budget-name budget-ec2-monthly --notification NotificationType=FORECASTED,ComparisonOperator=GREATER_THAN,Threshold=100,ThresholdType=PERCENTAGE --subscribers SubscriptionType=EMAIL,Address=<your-email-address>
Voila! We now have two AWS budgets configured in our account to notify us when our spend for either EC2 or RDS breaches the amount we expect.
Conclusion
Getting surprised by the cost of your AWS resources is not a great feeling. It's even less of a good feeling when the cost is unnecessary due to something like leaving a load balancer running. Therefore it is a good idea to put guardrails in place for yourself, even if your AWS account is just for personal use.
The first place to start is to have an account level billing alarm so you know when your monthly bill is going to be more than what you expected. The next step, if necessary for your work, is to do what we did here and add service level alarms.
Are you hungry to learn even more about Amazon Web Services?
If you are looking to begin your AWS journey but feel lost on where to start, consider checking out my course. We focus on hosting, securing, and deploying static websites on AWS. Allowing us to learn over 6 different AWS services as we are using them. After you have mastered the basics there we can then dive into two bonus chapters to cover more advanced topics like Infrastructure as Code and Continous Deployment.
Top comments (7)
Really one reason why AWS is so bad is it's pricing. Both Google Cloud and Azure are clearer on their pricing, and Google Cloud has the best pricing overall due to their automatic continuous usage discounts.
Why force the terrible system that is AWS on you, instead of looking on the alternatives? You'll get a more friendly interface, better pricing, and much more modern systems in general.
How are people ok with having to configure every VM instance separately on AWS still, for everything ranging from startup scripts, to proper hostnames and DNS setup, to alerting?
You have to be willing to commit a massive amount of manpower on writing automation systems around your cloud hosting to get even a remotely pleasant experience with AWS, and still you're stuck with the nonsense of their pricing models and just bad attitude.
I might just investigate alternatives, that would be a nicer experience.
That's definitely a fear of mine. Waking up in the middle of the night wondering if I left an expensive instance running. They won't stop you from running up a sky high bill, it's sort of "With great power comes great..." you get the idea.
Thank you for reminding! I just saw my billing dashboard and terminated everything used in the training video I watched.
Nice article Kyle. Cost is definitely one of the thing that catch people off guard when it comes to could usage; esp. when you leave an load balancer on overnight ;).
Thanks David. Leaving unneeded infrastructure running is definitely one of the leaders when it comes to unnecessary AWS costs.
That and security.
500 days old tokens, no mfa, etc etc...