AWS Cloud Financial Management
Finding savings from 2020 re:Invent announcements
This blog post is contributed by Stephanie Gooch, AWS Cost Optimization Lead
Over one hundred new services and features were announced in 2020 re:Invent (see the full list here). As customers, you can now select from an even wider range of resources to meet your needs. In this blog post, we will highlight three launches that can help you achieve cost savings.
New Amazon EBS general purpose volumes, gp3
Opportunity: Provides up to 20% lower price-point per GB than existing gp2 volumes
Along with up to 20% cost savings, gp3 volumes help you achieve more control over your provisioned IOPS, giving the ability to provision storage with your unique applications in mind.
To understand how you can leverage gp3 and the potential opportunity for cost savings, you will want to look into your current gp2 footprint. This information can be found in your AWS Cost & Usage Report stored in Amazon S3 (learn how you can set up your AWS Cost and Usage Report from this lab: Configure Cost and Usage Report Well Architected Lab). Using the query below in Amazon Athena, you will get a list of spend on gp2 volumes by volume resource ID for November 2020. This will then show you the new monthly price of the same size volume using the gp3 volume type and the amount you can save if you upgrade. You can also use the line_item_usage_account_id to find out which account has the most savings and this can prioritize what accounts to upgrade to gp3 volumes first.
Note: This query uses 0.088 gp3 pricing please check the pricing page to confirm you are using the correct pricing for your applicable region.
select * from (SELECT line_item_usage_account_id,
product_location,
product_region,
month,
pricing_public_on_demand_rate,
line_item_resource_id,
line_item_usage_type,
SPLIT_PART(SPLIT_PART(line_item_usage_type ,
':',2),'.',2) AS ebs_type,
sum(line_item_usage_amount) AS gb_charged,
sum(line_item_unblended_cost) AS sum_line_item_unblended_cost,
sum(line_item_usage_amount)*.088 AS gp3_price -- 0.088 eu-west-1 pricing
, (sum(line_item_unblended_cost)-(sum(line_item_usage_amount)*.088)) AS gp3_savings -- 0.088 eu-west-1 pricing
FROM "managementcur"."cur"
WHERE year = '2020'
AND month = '11'
AND product_product_name = 'Amazon Elastic Compute Cloud'
AND line_item_usage_type LIKE '%%EBS%%Volume%%'
AND product_product_family IN ('Storage','System Operation')
AND line_item_line_item_type = ('Usage')
AND product_region = 'eu-west-1'
AND SPLIT_PART(SPLIT_PART(line_item_usage_type,':',2),'.',2) = 'gp2'
GROUP BY line_item_usage_account_id, month, line_item_usage_type, product_location, product_region, line_item_resource_id, pricing_public_on_demand_rate, month
ORDER BY sum_line_item_unblended_cost DESC)
where gb_charged < 1000
Example output
One thing to note is that with gp3 you get a baseline performance of 3,000 IOPS, which is the equivalent of the number of IOPS you would receive for a 1TB gp2. Therefore the query above only looks for gp2 volumes that have less than 1TB provisioned storage. You should do more investigation for volumes over 1TB, if IOPS is key to the instance to which it is attached.
To find out the best way to modify your volumes to take advantage of the new gp3 type, take a look at this documentation: Requesting modifications to your EBS Volumes.
If you already have AWS Compute Optimizer enabled, you can read about the update that AWS Compute Optimizer now supports Amazon EBS volume recommendations.
Opportunity: Enables you to monitor, troubleshoot, and optimize the performance of AWS Lambda functions. Give you insights to find cost savings for Memory that has been over-provisioned.
When creating Lambda Functions, similar to EC2, customers may over-provision their memory, resulting in a higher fee per run. With AWS Lambda, you are charged by the number of requests for your functions and the time it takes for your code to execute, multiplied by the memory provisioned.
For example:
Lambda Cost = cost/request * [count(AWS_Lambda_Requests)] + cost/GB-second * [memory_size *AWS_Lambda_Duration]
CloudWatch Lambda Insights gives you access to the data used above to calculate your Lambda cost. To view this data, you will need to enable CloudWatch Lambda Insights by following the documentation: Getting started with Lambda Insights guide. There is a cost associated with these additional metrics and logs, see the pricing information on Amazon CloudWatch Pricing page and scroll down to Example 14 for more details. Once enabled, go to your CloudWatch Dashboard and select Lambda Insights > Performance Monitoring, and focus on the Memory Usage (Max) graph. This will show you which functions are enabled and the memory_utilization behind them. We suggest starting with any Lambda Functions showing under 50% max memory utilization, or you can look at what Lambda Functions cost you the most (use the lab: Well Architected Lab: CUR Queries Lambda query to see your monthly cost broken down by resource-id and find the Lambda functions with the most opportunity for cost savings.
Example:
The ‘SPTool_SPPricing_Download’ function is only using 37% of its provisioned memory, therefore it may be worth reducing the allocated memory. This will then lower the cost when it runs.
Amazon Athena engine version 2
Opportunity: There is now support for reading nested schema that can result in reducing cost.
The new Amazon Athena version is based on Presto (an open-source distributed query engine, to run queries) Version 0.217 and it includes several performance enhancements and new feature capabilities, such as federated queries, schema evolution support, additional geospatial functions, and support for reading nested schema to reduce costs. You can upgrade your Amazon Athena by following this information on this documentation: Changing Athena Engine Versions.
Athena charges for data scanned by the query. Consider a nested, column called Name that has <firstName> and <lastName> nested columns. In engine version 1, if you write a query to retrieve first name, Athena still reads the entire name column before filtering for firstName. In engine version 2, it will directly read only firstName, thus scanning less data, causing the cost to be lowered.
Alongside the new version, you can start Controlling Costs and Monitoring Queries with CloudWatch Metrics and Events for your Athena workgroup, by following these steps provided in this documentation: Enabling CloudWatch Query Metrics.
Then you can see how the changes to your Amazon Athena version impact usage and you can monitor your usage overall. By default, this is not enabled. Get this done early, especially if you are using Athena to query your Cost & Usage Report.
What to look for
Now you can view useful metrics for Athena and can start to look out for:
- Queries that are taking too long
- How much data they are scanning
- How frequently you are running queries (is this expected?)
If either of the first two points above seem high, then we recommend looking into what queries are running at that time. From there, you can potentially find ways to optimize them.
- If you are using ‘ORDER BY’ and you are just reviewing the top, use the LIMIT clause to shorten the data returned. This will distribute the load on the database.
- When you use ‘GROUP BY’ in your query, try and order the Group By’s from the largest list of options in the group, to the smallest
- If you have partitions, utilize them to reduce the amount of data that is scanned causing the cost to be lowered
For more information on optimizing your Athena usage, check out this blog post Top 10 Performance Tuning Tips for Amazon Athena.
Conclusion
Hopefully this blog has provided you some information on how to save costs with the newly released services and features from re:invent 2020. Read more 2020 re:Invent announcements from this blog post AWS re:Invent 2020 – Top Announcements. You can visit our Cloud Financial Management web portal and blog channel to stay up to date with launches, best practices, and successful customer stories.