Optimize and save on “other” services

Inspired by Steph Gooch’s 2022 re:Invent session “Optimize the others” chalk talk session

If you can’t measure it, you can’t manage it. In order to optimize your cloud resources efficiency, you need to be able to view and understand cloud cost and usage data. Often times, you focus on the top spenders, because they drive up your bills and are easy to identify. For example, when filtering your AWS cost by service in AWS Cost Explorer, you usually see the cost breakdown by services, such as, Amazon EC2-instances, Amazon Simple Storage Service (S3), and Amazon Relational Database Service (RDS). The rest of the services are lumped together in a category called “Others” in the graph. However, the cost of the “Others” category can be just as high as the top cost drivers. It’s worth looking into the sources of these costs and identifying opportunities for cost and performance optimization. In this blog, I’ll use a few examples to demonstrate how you can dive deeper and understand the cost elements of these “other” services and what you can do to optimize the spend.

Speaking of the services that usually fall under the category of “Others”, I want to start with services that are labeled as “EC2-Other”. “EC2-Other” tracks costs associated with the Amazon EC2 instances, such as, Amazon EBS volumes and snapshots, elastic IP address, data transfer, NAT gateways, and more. You can view the details of your “EC2-Other” spend by grouping with “usage type” in Cost Explorer. Services, such as, AWS Compute Optimizer and AWS Trusted Advisor, offer recommendations for EBS volume usage, including detecting unattached EBS volumes, and migrating from one volume type to another, e.g. General Purpose SSD (gp2) to gp3. EBS snapshots back up data from your EBS volumes and are charged as incremental data changes made to your EBS volumes. To save costs on EBS snapshots, you can start with single snapshots that were created with your AMI (Amazon Machine Image). When creating an AMI for an existing EC2 instance, Amazon EC2 creates EBS snapshots of your instances’ root volume and other EBS volumes attached to your instance. However, when you deregister your AMI, but did not opt to delete the snapshot, that snapshot will persist and cause full storage charge. Make sure you deregister the AMI and delete the associated snapshots, when you no longer need them. You can automatically create, retain, and delete EBS snapshots and EBS-backed AMIs with Amazon Data Lifecycle Manager.

Now, let’s talk about services that are categorized under “Others”.  Services, such as Amazon CloudWatch, AWS CloudTrail, and AWS Lambda, are usually on the list. CloudWatch monitors the performance of your AWS resources and applications run on AWS. CloudWatch metrics enable many other AWS services, such as AWS Compute Optimizer, to provide optimization recommendations. However, CloudWatch itself incurs costs, if your usage exceeds the free tier limit. For instance, many AWS services send basic monitoring metrics to CloudWatch at no charge. The basic monitoring is enabled by default once you start using these services. But if you choose to enable detailed monitoring for services that offer the option, for example, sending EC2 metrics more frequently at one-minute intervals, instead of the five-minute intervals used in the basic monitoring, you’ll incur EC2 detailed monitoring cost (typically charged at $2.10 per instance per month and goes down to $0.14 per instance at the lowest priced tier). If you no longer require detailed monitoring metrics for your EC2 instances, you should turn off the detailed monitoring. If you’ve set up a CloudWatch alarm that is associated with multiple metrics, you should be aware that each alarm-metric incurs cost. Furthermore, CloudWatch logs, custom and vended logs, are other resource usage type that can incur ingestion charges based on the volume of the ingested/scanned log data, or archival charges, if you forget to delete data after certain retention period (the default retention policy on log groups is set to “Never Expire”). Actively manage how your CloudWatch detailed metrics, alarms, and logs are configured, based on your applications’ needs, so you can reduce your CloudWatch costs. Learn ways you can reduce your CloudWatch charges from this knowledge center article “How can I determine why I was charged for CloudWatch usage, and then how can I reduce future charges?”

As CloudWatch tracks the performance of AWS services and resources in your environment, CloudTrail monitors the API activities made in your AWS environment for governance and auditing purposes. The service is free to use, when there is only a single copy of management events delivered via trail to your S3 bucket. However, several use cases will incur cost, for instance, if you create additional copies of management events at the organizational or account level. CloudTrail also incurs costs when you record and analyze events using CloudTrail Lake, or choose to analyze events using CloudTrail Insights. Due to the fact CloudTrail is enabled by default, it is possible that additional copies are created at the organizational and account level. One way to identify any duplicated management event records is to look for the “PaidEventsRecorded” metric under your CloudTrail service charges on your AWS bill to see if there are additional copies of management events recorded in any specific regions (Image 1). For intentional control of what management and data events to log, you can use CloudTrail Event Selector and Advanced Event Selector to specify the logging criteria. Work with your Cloud Center of Excellence Office to centrally manage CloudTrail and identify who owns duplicated CloudTrails. If you decide to remove additional trails, make sure you delete the S3 buckets and objects attached to it, so you are not continuously paying for logs that you no longer need.

Image 1. CloudTrail service charge on AWS Bill

Many of you are aware of the cost saving advantage by leveraging serverless applications and the gained operational efficiency for the ongoing maintenance and infrastructure. AWS Lambda, the serverless, event-driven, compute service, allows you to run code at scale without provisioning or managing infrastructure. Yet, it’s helpful to understand the key components of its pricing structure, for example, the number of requests, duration of invocations, and the configured memory allocation. You can optimize the memory configuration of Lambda functions with AWS Compute Optimizer or AWS Lambda Power Tuning. To increase the efficiency of the invocation, besides applying best practices to your Lambda code, you can set realistic function timeout to prevent prolonged invocation. Choose Graviton2 architecture for arm64-based infrastructure and gain up to 19 percent better performance at 20 percent lower cost. For steady Lambda workloads, consider Compute Savings Plans for discounted rates of up to 17% for a 1 or 3 year term, and enjoy the flexibility to apply the benefits across EC2 and Fargate. Understand what its pricing is comprised of and leverage services, such as, Compute Optimizer, for usage optimization, and purchase option (Savings Plans) for discounts. Learn about other techniques to reduce your AWS Lambda costs from this blog post.

Conclusions

In the blog post, I used a few examples to show how you can zoom into the services that typically fall under the “others” category and understand their cost elements and ways to drive down the cost. It’s helpful to have dedicated FinOps professionals to frequently audit the usage efficiency and work across teams to implement necessary changes. Hope the blog post can help you get started and discover cost saving opportunities that fit your application and business needs.