DEV Community: Viraj Nadkarni

Practical considerations with commitment based Pricing models on AWS

Viraj Nadkarni — Tue, 11 Jun 2024 06:45:21 +0000

Pricing of services on AWS cloud comes in different flavors such as free tier, pay as you go, spot and finally commitment based pricing models such as reservations and savings plan.

The commitment based pricing models can lead to considerable savings, if utilized optimally for the right use case. However, it is important to understand the practical considerations when opting for this pricing model.

This article summarizes few of the key tips to keep in mind if planning to opt for this pricing model. Please refer to my whitepaper for more details on each of the below items.

Over-assuming savings percentages

The highest savings percentages often comes with riders such as longer commitment periods, stringent usage terms such as being restricted to only a particular service type and higher upfront payments. Be pragmatic with your organization's risk appetite and savings that can be obtained based on that constraint.

Crossover points vs breakeven point

You may be asked to present a forecasted pricing plan and savings when proposing this pricing compared to a pay as you go model. It is important to understand when the crossover happens and when the actual breakeven occurs and not confuse the two.
Crossover point is when the pricing of the commitment based model matches the pay as you pricing and subsequently gets cheaper. The catch here though is you have still committed to a set usage period in case of commitment based pricing whereas in pay as you go pricing, you are under no such restriction to continue using. An actual breakeven point calculation will need to take this into perspective.

Explore alternative commitment options

For shorter term commitments, there are sometimes better alternatives than directly opting for a reservation or savings plan model, both of which come with a minimum of 1 year commitment such as exploring the reserved instance marketplace. An example is purchasing off the reserved instance marketplace where there is an instance available with lesser commitment period remaining. Explore these alternatives beforehand.

Thoroughly understand your workload patterns

The Savings plan option is usually the more flexible of the two options but that does not mean it is also always the right option. It may turn out expensive for certain patterns and also if you need to cancel or dispose of the commitments at short notice. For example, With reserved instances you have option to sell them off on the marketplace but not with savings plan. Be sure to weigh the pros and cons of both options for your workloads.

Understand price calculation difference

The Savings plan bills you on hourly commitments whereas Reservations are billed on an a monthly basis. Whereas this may not seem to be big deal at first glance, it can very well turn out to be one when there are large fluctuations in workload usage. A key point to understand here is that with savings plans, if your workloads consistently under utilize the committed savings plan amount, the loss in commitment amount cannot be adjusted in retrospect in subsequent period maybe for some spike usage in future.
So analyze your workload usage patterns before opting for either options.

Miscellaneous considerations

Other than the above tips, there are also items to look at. Some of these include,

Pricing changes of the underlying services
Cloud providers such as AWS regularly come up with price reductions for their services. For example an EC2 instance type that cost x USD/hour sometime back might now cost .95x USD/hour in pay per use mode. So if you have calculated the savings of commitment based pricing over pay as you go assuming x USD/hour as the pricing for pay as you go, that calculation needs to be reworked.
Calculate savings based on practical usage time
When comparing commitment based pricing with pay as you go model, understand that with pay as you go model, you are under no restriction to use the service, 24x7. But with most commitment based pricing, you sign up for a fixed hourly or monthly commitment price, irrespective of usage.
So account for this when comparing these pricing models. Also, below a certain threshold of usage, it may not make financial sense to commit to a commitment based pricing so understand your practical workload usage patterns.
Introduction of newer instances at better price points
Cloud providers regularly introduce a new class/series of instances, at better performance but lower price point than the existing instance class. Based on how flexible the commitment option is, you may be able to take advantage of this by switching to the newer class of instance.
Centralizing your reservation strategy
For multiple teams and accounts, centralizing the process will lead to better usage visibility across teams and greater organizational efficiency.

To conclude, it is important to assess and plan out your commitment based pricing journey carefully, not over-commit initially and adaptatively optimize the commitments made based on past and forecasted usage data.

Optimizing Data Management on AWS - Part 2

Viraj Nadkarni — Tue, 11 Jun 2024 05:31:05 +0000

We started out with introduction to Data management strategies on AWS in Part 1. Here are some additional strategies to look at.

Use policies to manage lifecycle of your data

Data has a lifecycle, just like everything else, and you need to have a plan to control it from the time it was first created to the time it is archived or deleted. As the data moves through this lifecycle, its storage requirements often change. If you are not managing the lifecycle of your data, chances are that your data is being stored in costly or inefficient storage. The recommended approach here is to first identify the lifecycle pattern for your data and then use automated lifecycle polices to manage the lifecycle of these datasets.
Doing so will ensure that data is stored in the most appropriate storage tier at each stage of its lifecycle. Note that the lifecycle management evaluation should include areas like understanding your data characteristics, data access patterns at each stage of data lifecycle, handling data that is old or rarely used, archival and finally data deletion.

Get rid of redundant or unneeded data.

Just as keeping underutilized or idle resources running on cloud costs money, so does data. Storing redundant or unneeded data not only consumes unnecessary storage resources but also increases costs. By removing such data, organizations can free up valuable storage resources and reduce their environmental impact. This problem often manifests in different forms such as data being unnecessarily backed up, duplicated or stored redundantly irrespective of its criticality(touched upon in item 2) or when the data itself is easy to recreate if the need arises.

Monitor data movement to reduce costs

Monitoring, optimizing and minimizing data movement across networks can help with reducing the overall resources need for supporting data movement and indirectly reduce your overall costs besides helping in other areas such as performance.

Ask yourself, have you considered proximity of data or users of your workload when selecting a region on where to store data ? Are you leveraging services such as Lambda@Edge or CloudFront Functions that help you run data closer to your users ? Is the serving of data itself optimized ? Is the data being served up in efficient file formats or compressed ? Is the data being moved in line with your business needs ? Have you evaulated that only relevant data and too in the level of granularity that is needed by your application is being passed around?

To conclude, data management plays a pivotal role in an organization's sustainability journey. By avoiding these anti-patterns and implementing the associated best practices, organizations can ensure that they are using their resources efficiently, and thereby reduce the costs and minimize environmental impact.

New Launch - AWS Free Tier API

Viraj Nadkarni — Tue, 28 Nov 2023 06:18:50 +0000

Like other cloud providers, AWS provides free tier pricing to specifically allow users the ability to try out its services up to a certain limit.

Ignoring some services that are always free such as AWS IAM,
few considerations to note with the free tier model is that not all AWS services are covered under the free tier model.
For the ones that are covered under this pricing model, the free tier structure varies. For example,
a) there are some that are free for a longer duration (typically 12 months) up to a specified limit (e.g., AWS S3 is free for up to 5 GB usage with get/put request limits),
b) there are others that are always free services up to a certain limit usage (e.g., Azure functions with a million executions free per month) and
c) then there is another category that is free upto a certain shorter duration and/or credit limit. example, Amazon Redshift.

Once free tier exhausts, you are switched to pay-as-you-go pricing. This was often a source of confusion in practice for newbies who often do not have proper cost governance in place as it would be difficult to track when the transition to a paid pricing model has happened. Not that this information was entirely unavailable but it had to be figured out using the AWS billing and cost management service. Zero spend budget template is a handy way to know when you have moved out of free tier and the AWS free tier under the cost analysis section in Billing and cost management provides the current and forecasted usage of free tier eligible services.

AWS has now released the free tier API , which provides an alternate way to tackle this problem by providing details such as the actual consumption, the forecasted consumption for the billing period and the limit of the free tier usage. If the forecasted consumption is greater than the free tier limit, then it means that you will incur some cost if the service consumption continues at the same rate.

For now, this is available only using the SDK or CLI and any reporting, automation and alerting needs to be custom built.
A caveat - dont quote me on this as I haven't exhaustively used this API:). Like the free tier item in Billing and cost management, this API returns list of services that are eligible for some free tier usage in your current account. Ones that are not eligible, for example those that come under free tier pricing only for new users for specified duration (first 12 months after account creation as in case of AWS EC2) may not appear in the list based on how long this account has been in use. So, do not restrict monitoring of your AWS usage outside the free tier to only using these techniques.

Optimizing Data Management on AWS - Part 1

Viraj Nadkarni — Tue, 07 Nov 2023 05:14:43 +0000

Introduction

In the cloud setup, having a good data management plan on Cloud is of utmost importance for multiple reasons. Reasons for this are many. Cloud costs are typically linked to usage and as such poor data handling can result in rising costs due to unnecessary or outdated data. Further, since cloud data might cross several regions, it's essential to adhere to different regulations and uphold strict security standards, which includes measures like encryption and controlling access.

This blog touches upon 5 critical areas you should be looking at with regards to data management. This part highlights the first two, Part 2 covers the remaining three. For more exhaustive details, please refer to the AWS well architected whitepaper.

Do not assume Uniform Data Storage and Access Patterns

Too often, we assume that all data storage can be managed uniformly in the same storage type. But for multiple workloads of any sizeable complexity, this is not likely to be the case. Every workload has its unique data storage and access requirements. Hence assuming that that all workloads have similar patterns or using a single storage tier for all workloads can lead to inefficiencies. The sooner these patterns are recognized the better. The advantage of recognizing and catering to these patterns is that it reduces the resources required to meet business needs, and thereby the overall efficiency of the cloud workload. To address this issue, regularly evaluate your data characteristics, your access patterns and plan to migrate data to the appropriate storage type that best aligns with these patterns. Also, understand that this will not be a one time activity but an exercise that needs to be conducted regularly.

For a comprehensive evaluation, the decision guides provided by AWS for storage and database services should be a good starting point.

Have a solid data classification strategy

Data classification is the process of categorizing data based on its sensitivity and criticality. A common mistake is not identifying the types of data they process and stored based on its sensitivity and importance. This ends up being a massive oversight and not having a classification strategy could lead to other consequences such as inappropriate security controls in place and even lead to compliance, regulatory or legal issues.
By having a proper data classification policy, organizations can determine the most performant and cost optimized storage(and even energy efficient, if sustainability is one of the key drivers) tier for their data. To come up with a data classification strategy, one should conduct an inventory of the various data types and then determine their criticality. Also have a periodical audit the environment not just for untagged and unclassified data but also to re-evaluate the data classification conducted earlier to see if that needs any change as per changing business conditions.

AWS Data classification guide

Few other strategies are covered in Part 2 of this blog.