Bob had just arrived in the office for his first day of work as the newly hired chief technical officer when he was called into a conference room by the president, Martha, who immediately introduced him to the head of accounting, Amanda. They exchanged pleasantries, and then Martha got right down to business:
"Bob, we have several teams here developing software applications on Amazon and our bill is very high. We think it's unnecessarily high, and we'd like you to look into it and bring it under control."
Martha placed a screenshot of the Amazon Web Services (AWS) billing report on the table and pointed to it.
"This is a problem for us: We don't know what we're spending this money on, and we need to see more detail."
Amanda chimed in, "Bob, look, we have financial dimensions that we use for reporting purposes, and I can provide you with some guidance regarding some information we'd really like to see such that the reports that are ultimately produced mirror these dimensions — if you can do this, it would really help us internally."
"Bob, we can't stress how important this is right now. These projects are becoming very expensive for our business," Martha reiterated.
"How many projects do we have?" Bob inquired.
"We have four projects in total: two in the aviation division and two in the energy division. If it matters, the aviation division has 75 developers and the energy division has 25 developers," the CEO responded.
Bob understood the problem and responded, "I'll see what I can do and have some ideas. I might not be able to give you retrospective insight, but going forward, we should be able to get a better idea of what's going on and start to bring the cost down."
The meeting ended with Bob heading to find his desk. Cost allocation tags should help us, he thought to himself as he looked for someone who might know where his office is.
In this fictitious story, Bob needs to figure out where these expenses are coming from so that he can start to work on lowering the cost. The AWS cost explorer, while helpful, on its own is simply not enough in this case. Bob needs more detail, and that's where cost allocation tags (CATs) can be helpful. There are other tools in AWS that can assist Bob with this task, and we'll mention those in brief detail later in this article.
In general, tags have some characteristics which we should be aware of (and if I missed something, please add your comment):
- Tags can help demonstrate relationships between resources.
- Tags are metadata about your resource, and these simple key-value pairs can be added to just about any resource in AWS.
- Tags are ideally a single key and a single value juxtaposed with a single key and multiple values.
- Tags are limited to 50 per resource.
- Tags can have rules enforced on them, which helps to ensure consistency and also meet compliance requirements.
- Some tags are also generated automatically by AWS (
aws:createdBy, for example) — these tags are immutable. - In addition to cost allocation, tags can be used for automation, operations, and access, as well as security risk management.
- As it pertains to CATs, tags can be useful for identifying cost center, business unit, department, project, product, geographic area, or purpose, for the purposes of financial reporting.
- Tags only appear in reports at the time the tag is created — there is no backdating.
In the next section, we'll look at the AWS billing dashboard, where we can activate a tag, and also at a report that's filtered by that tag.
In this section, we're going to look at the AWS billing dashboard, which is available to account holders and which has two items of interest: the cost explorer and cost allocation tags. We can see both in the next image.
In the image below, an inactive cost allocation tag will be activated and then a refresh request will be executed. The comment in the image mentions tagging an S3 bucket — we can tag any resource in AWS.
The following image demonstrates how we can filter by tag in the AWS cost management cost explorer AWS web UI.
If we filter by costCenter tag 12345, we can get an idea below what costs are associated with this key and value, specifically.
In the next section, we'll go over three examples pertaining to how resources, such as an EC2 instance, in this case, can be tagged with two tags via the web interface, via the command line (CLI), and programmatically.
In this section, we will review three examples of how CATs can be applied to an AWS EC2 instance on creation. We will cover tagging via the web user interface (UI), via the AWS command line, and finally, we will demonstrate how an EC2 instance is tagged at creation time using Node.js.
Tagging an EC2 instance via the web user interface (UI)
Creating an EC2 instance via the AWS web interface is easy enough. We can see below that we can add tags in step five.
We've skipped ahead in this example, and below you can see step five, where we've assigned costCenter and department key-value pairs (KVPs) to this instance.
Once the instance has been launched, we can see that the tags we've assigned above are assigned to the instance itself.
Tagging an EC2 instance via the command line interface (CLI)
This example is fairly straightforward. The steps not included here involve setting up the profile, which requires creating a user and assigning the appropriate permissions, as well as creating the VPC.
We also need to create a profile in the /Users/[user]/.aws/credentials file, which includes the aws_access_key_id and aws_secret_access_key with AWS-provided values. This is necessary for both the example below and the programmatic example which follows.
The following script will create a t2.micro instance in the us-east-1 region and deploy it in the subnet-0969b587cc72969d2 subnet. Most importantly for this example, the EC2 instance will be tagged with the costCenter and department KVPs.
aws ec2 run-instances --image-id ami-02354e95b39ca8dec --count 1 --instance-type t2.micro --region us-east-1 --profile thospfuller-aws-cli --subnet subnet-0969b587cc72969d2 --tag-specifications 'ResourceType=instance,Tags=[{Key=costCenter,Value=45678}, {Key=department,Value=Energy}]'
The aforementioned subnet value can be found under the VPS Subnets option as shown in the image below.
In the following image, we can see the costCenter and department tags appear on the EC2 instance.
This brings us to the last example that we'll cover here, and that is to launch programmatically an EC2 instance that's been tagged with the appropriate KVPs.
Tagging an EC2 instance programmatically
Similar to the previous example, this example is fairly straightforward. The steps not included here involve setting up the profile, which requires creating a user and assigning the appropriate permissions, as well as creating the VPC.
We also need to create a profile in the /Users/[user]/.aws/credentials file, which includes the aws_access_key_id and aws_secret_access_key with AWS-provided values.
The following will create a t2.micro instance in the us-east-1 region, deploy it in the subnet-0969b587cc72969d2 subnet, and, most importantly for this example, tag it with the costCenter and department KVPs.
Line #49 has the costAllocation and department KVPs. Note that the gist below can be found on GitHub and the file is also available in the following repository.
In the following image, we can see the output when the script is executed, along with a pointer to the costCenter and department tags. Note the instance id is just above the red arrow pointer which points to the tags.
In the following image, we can see the instance id as the EC2 instance is starting.
Finally, we can see the costCenter and department tag keys and values in the image below.
In the next section, we will discuss AWS Organizations, AWS Config, and how these services can help a business achieve tag compliance.
Two choices are available to businesses that are relying on CATs to ensure consistent application and format of tags assigned to resources: AWS Organizations and AWS Config_._ Since one can easily learn about these directly from the source, we only provide a brief description that will help the reader see the differences:
- AWS Organizations —AWS Organizations is for account management and can consolidate multiple AWS accounts under one organization that is managed centrally; the AWS Organizations service also helps with budgeting, security, and compliance.
- AWS Config — AWS Config is also a service that helps the user with assessing, auditing, evaluating, securing, and troubleshooting the configuration of AWS-specific resources. If your business is not using AWS Organization, then AWS Config can act as an alternative for managing tags; see also Tagging Your AWS Config Resources.
Without auditing and enforcement, the business will easily end up with a mess, so it's important to not just use the tools available to better understand where the money is going but also to ensure that all resources are created with the appropriate tags in the appropriate format. Keep in mind that engineers have to apply these changes, and without enforcement and compliance, they simply won't do it or do it consistently and correctly [ 7 ].
In our fictitious story, not only will Bob want to take advantage of CATs, but he'll also additionally benefit from utilizing either AWS Organizations or AWS Config, especially seeing as he's in charge of overseeing two divisions with 100 software engineers and support staff.
It is beyond the scope of this article to demonstrate how these two services work; however, we may delve into this in detail in a future article.
So far we've introduced tags and cost allocation tags, we've demonstrated how tags can be added to a resource such as an EC2 image, and we've reviewed how one can go about ensuring that compliance measures are in place to ensure that the tag keys and values adhere to an expected format.
The last piece of the puzzle that we'll discuss in this article involves tag strategies. It is not enough to allow engineers and managers to add tags with known values to AWS resources they're using. In fact, in an organization the size of the one that Bob runs, this would be exactly what you'd want to avoid because the business has multiple divisions and many software engineers and support staff. In this case, a tagging strategy needs to be developed with input from the people managing these divisions, with the CTO at the top ensuring that tags are uniform across the business and that there is a minimum set of tags defined, along with optional tags which are applied to every taggable resource in use across the business.
Lastly, when it comes to cost allocation tags (CATs) specifically, our CTO Bob will want to ensure that both a minimum set of required and optional tags are defined and that appropriate key-value pairs are married with financial reporting dimensions, which will be provided to us by, in this story, the head of accounting, Amanda.
In this article, we introduced cost allocation tags, demonstrated how they work, and reviewed two services, AWS Organizations and AWS Config, which can be used to achieve tag compliance across all AWS resources. These are not the only tools at Bob's disposal for getting the cost of AWS usage in the business under control. In fact, the addition of any of the following should prove to be helpful as well:
- Cost management
- Budgets
- Billing alarms
- Trusted Advisor
- Third-party cost optimization software
- Continuous improvement strategy [15, 16, 17, 18]
- Developing a performance-optimized architecture 13
We concluded the article with a basic discussion of tagging strategies, which are very important and which need to be given business-wide attention when defining a set of required and optional tags for use across the business and for cost allocation purposes.
Our budding CTO Bob will be off to a good start implementing tag strategies in the business, but what would you do? Let me know in the comments, right after you finish taking the quiz below.
You thought you were done, didn't you?
That's right, get out a blank piece of paper and a pen because I've included a pop quiz. If you don't have a paper and pen available, place your answers in the comments section for discussion.
- Find three other resources that can be tagged. (This is easy)
- What resources cannot be tagged? (This is harder, see 10, and don't assume it's correct or complete.)
- Developing a cost-optimized architecture is one of the five pillars of a [fill in the blank]. (Hint "AWS [followed by three words].") 8
- Name the other four pillars from 2. without looking at notes.
- What is the difference between AWS Organizations and AWS Config?
- In our fictitious story above, what would be the more appropriate service (AWS Organizations, AWS Config, or both) for the business he works for? Why?
- Can a business use both AWS Organizations and AWS Config?
- What other pillar complements the cost-optimized pillar? (There's a specific pillar with a very specific reason.)
- Tagging Best Practices: Implement an Effective AWS Resource Tagging Strategy
- AWS Tagging Best Practices Guide: Part 1 of 3
- AWS Tagging Best Practices Guide: Part 2 of 3
- Deactivating the AWS-Generated Cost Allocation Tags
- How to reduce your AWS costs? Save up to $500k with these guidelines!
- Cost optimization for Kubernetes on AWS
- AWS Tagging Best Practices — The Ultimate Guide
- Best Practices for Organizational Units with AWS Organizations
- The 5 Pillars of the AWS Well-Architected Framework
- AWS Spending That Can't Be Tagged
- AWS Config and AWS Organizations
- Managing AWS Organizations accounts using AWS Config and AWS CloudFormation StackSets
- Performance Efficiency
- Optimizing your cost with Rightsizing Recommendations
- (Still) learning from Toyota
- Creating a Culture of Continuous Improvement
- The Kaizen Approach to Software Development
- Why Continuous Improvement Is Worth It?
Top comments (0)