Category: AWS Well-Architected Framework
Cloud providers like AWS have already made lives easy by taking all your infrastructure-related worries away. This comes at a cost, but it is still very cost-efficient. Moreover, talking about costs — just doesn’t stop here. As they say, there is always scope for improvement. In this post, we take a look at some ways by which we can optimize cloud costs even further, in the context of AWS.
Before we take a look at any specific service or strategies, let us take a moment to understand the importance of tags. To deal with costs and answer the expenditure-related questions, tagging is highly recommended — it is a best practice. Tagging of resources allows us to have greater visibility and granularity over cloud expenditure.
As an organization, standard tagging formats should be defined and enforced while creating cloud resources. The format may define aspects like the project, portfolio unit, teams, etc at an organizational level. Going deeper, project-based conventions may as well be defined to represent services supported by various cloud resources. However, we have to be conscious about how many tags to choose — there shouldn’t be too many or too few tags defined. In general, tags can be categorized as below:
*Technical *— representing technical details like automation, security, etc.
*Strategic *— representing organizational details like a cost center, access control, governance, etc.
How do we track costs? AWS provides a few highly useful services like Cost Explorer to explore your cloud spendings over time. It offers a nice visual interface representing monthly or daily costs. It provides us with a default dashboard representing the monthly costs incurred per service. AWS Cost Explorer allows us to generate and export cost reports at a high level as well as granular and specific reports. You can build your reports and dashboard as per your interests and focus.
Cost Explorer helps you set budgets which helps monitor the costs. Budgets are a great way to keep your costs under control. Using budgets you can define expenditure baseline in AWS, and threshold breach notifications can be configured. For example, if the costs exceed more than 80% of the baseline budget, we can opt for receiving a notification so that the alert mechanisms help us take action. Cost Explorer also provides us with rightsizing recommendations. Recommendations help us identify the scope where we may be provisioning more than required infrastructure in terms of instance type, pricing model, etc.
If we need a more detailed reporting tool, AWS offers QuickSight service. It is a business analytic solution for cost reporting. It is fast and highly scalable and includes ML capabilities. We can explore, analyze and collaborate on cost expenditure topics in a much better way. However, this is not a free service like Cost Explorer and it works on a pay-per-session basis.
Cost Optimization is one of the pillars of the AWS Well-Architected Framework. **AWS Trusted Advisor **is a service that embodies a virtual service from AWS that advises us about this framework. It performs a series of checks with AWS best practices and highlights them in the below format if any actions are required.
no problem detected — meaning implementation is as per the required standards.
investigation recommended — for warnings.
action recommended — for any aspect which is totally out of place.
AWS Trusted Advisor continuously monitors the usage of resources provisioned and generates recommendations. In the case of Cost Optimization, it highlights if any resources are underutilized, if instances are idle, reserved instance expirations, etc.
We do not always have to wait for AWS Trusted Advisor to advise on optimizations before we take action. **AWS CloudFront **is a service that provides resource metrics using which we can monitor performance ourselves and identify underutilized resources. AWS CloudWatch is the easiest way to collect metrics since it integrates with several AWS Services directly. By gaining operational visibility and insights, we can act on improvements and optimize costs.
Right-sizing of compute and storage resources is a very important aspect as the majority of the costs are associated with these types of resources. AWS offers various options to provision a virtual machine on their infrastructure. These options are created to suit your need as per the criticality of your service.
In any given implementation, not all the services require dedicated high-performing nodes. Similarly, not everything can work on low compute and less available nodes. This provides us with a gap to explore and define our compute infrastructure that is most suitable for the business and most cost-efficient. Let us take a look at some of the compute types (EC2) provided by AWS.
*Reserved Instances *— long-term commitment, low cost.
*Spot instances *— very low cost, uses sparing EC2 capacity, released when capacity not available, good for fault-tolerant applications.
*On-Demand instances *— no commitment, regular costs.
*Dedicated instances *— instances created from resources that are not shared.
*Dedicated hosts *— dedicated instance with access to hardware options like ports.
*Reserved capacity *— Reserved capacity can be purchased and used within an instance family. Instances can be resized, based on the normalization factor. Helps reduce cost with flexibility.
Based on the availability requirement, we can select appropriate options from above to host our workload. For example, when we are sure about a certain node that exists for the long term, we can take advantage of reserved instances instead of On-Demand instances and save up to 75% of the costs. There is no point in provisioning an On-Demand instance for loads that are ephemeral and non-critical. Spot instances can be used in this case that can help reduce costs up to 90%.
Similar strategies apply for storage resources in AWS. AWS provides various types of storage and we can use appropriate storage based on how hot or cold we want the data to be stored. Various types of storage offered by AWS are — Object, Block, File, Hybrid, Edge, and Backup.
Considering an example of Object storage — below are storage classes offered by AWS S3. Which object storage class should be chosen, depends on how frequently the data is accessed, what is the retention period. Lifecycle policies can be used to transition the old data to cheaper long-term storage.
*Standard Storage *— standard storage, regular costs, immediate access.
*Standard Infrequent Access *— reduced availability, reduced costs.
*One Zone Infrequent Access *— reduced redundancy, reduced costs.
*Intelligent Tiering *— for unknown access patterns, data is moved in and out of various classes based on file usage frequency.
*Glacier *— Long term storage, cheap, minutes or hours to retrieve.
*Glacier Deep Archive *— Longer-term storage, cheaper, hours to retrieve.
Every type of storage comes with various levers which can be set appropriately to optimize your storage costs on AWS. It is recommended to make use of AWS Data Lifecycle Manager while provisioning your storage capabilities.
Cost optimizations are applied to resources in AWS and how to apply these optimizations depend on the business priorities. Mainly we need to decide if our focus is on costs or on time-to-market. Aspects of cost optimization can be categorized below categories:
**Time-based — **to optimize over time.
*Demand-based *— to optimize based on demand/traffic.
*Buffer-based *— to optimize based on secondary workloads.
In general, the cost optimization pillar of a well-architected framework suggests that while designing, developing, and deploying applications on AWS it is a good practice to keep cost optimization in perspective. It should be a continually monitored process to reap the most benefits from the cloud investment.
Hey, if you like this content, do consider subscribing, following, and sharing this blog post! Let'sDoTech, Instagram, Twitter, LinkedIn
Originally published at http://letsdotech.dev on March 4, 2021.
Top comments (0)