How to use AWS Well-Architected with AWS Trusted Advisor to achieve data-driven cost optimization

Are you looking for ways to optimize your costs on AWS? Are you ensuring that you are taking advantage of all the cost-saving features and services that AWS offers? If not, you should be! In this blog post, we will discuss how to use AWS Well-Architected  and AWS Trusted Advisor to achieve data-driven insights that can help you achieve your cost optimization goals.

Let’s begin by taking a closer look at the AWS Well-Architected Framework. It is a set of architectural and operational design principles across six pillars that was developed by AWS to help organizations build secure, high-performing, resilient, and efficient infrastructure of applications or workloads. The framework contains best practices for implementing the design principles. These best practices are grouped under a set of questions for each pillar, which can be accessed through the AWS Well-Architected tool (AWS WA Tool). The tool can then be used to drive a discussion-driven AWS Well-Architected Framework Review (WAFR) which helps in evaluating what best practices are in place, and where improvement opportunities are for the workload. Potential risks are identified in the process, and are categorized as High Risk Issues (HRI) or Medium Risk Issues (MRI) based on their severity.

The Cost Optimization Pillar is one of the six pillars of AWS Well-Architected Framework that is commonly a focus during WAFR. The guidance in this pillar helps customers build and operate workloads with cost-awareness in mind. Cost optimized workloads will help customers achieve business outcomes while minimizing costs, allowing organizations to maximize their return on investment. It contains design principles that provide high-level goals, definitions and patterns to achieve best practices from a cost perspective.

During the WAFR, using data points observed from the environment to evaluate which best practices are in place will help reduce subjective bias. You can leverage several AWS services, like Trusted Advisor, to collect these data.

Let’s explore more on how you can run a data-driven approach in running WAFR for cost optimization by leveraging Trusted Advisor and AWS WA Tool.

Cost Optimization in AWS Well-Architected Framework Review

In the AWS Well-Architected Cost Optimization Pillar, we have five design principles, five best practice area definitions and ten questions defined for improving our cloud spending posture. Let’s look at COST.6 as an example, this question usually points out unnecessary spending caused by lack of awareness or misconfiguration.

COST 6, “How do you meet cost targets when you select resource type size and number ?”

Figure 1.  A screenshot of the AWS Well-Architected Tool. Question Cost.6 shows the question title and description for reviewer to lead the discussion with workload sponsor and pillar sponsors

This question wants to ensure that the workload uses the most cost-effective type, size and the number of resources.

In the AWS Well-Architected Cost Optimization Pillar, this question is categorized under “Cost Effective Resources” best practice area. And the best practice items under this question refers to the “Right Sizing” practice that is needed around your workload to ensure cost efficiency. This practice supports the workload in achieving “Adopt a Consumption Model” design principle of Cost Optimization Pillar. Here is the end-to-end chart from pillar to best practice to give you better visualization:

Figure 2.  A diagram to explain the structure of AWS Well-Architected Framework, from pillar, design principles, definition, question and down to best practice.

Data driven review using insights from AWS Trusted Advisor

While running WAFR, certain areas in the conversation require preliminary data point before any tangible outcome can be achieved. This often leads to a longer path to resolution. The conversation can either leads to further discovery activity or work items that are non-directly resulting the intended outcome.

In the context of Right-sizing for computing resources such as AWS Lambda function, you need to monitor the utilizations of the function over a period of time to determine how much a memory allocation is most cost-optimized. In the absence of possible observation references, the WAFR conversation tend to result in collecting metrics data or enabling monitoring capabilities. A few steps away from the intended outcome, which is configuring the memory allocation to a more optimal size.

Figure 3.  Example scenario: in a review conversation which sponsors does not have enough data point to support the decision making

Amazon CloudWatch logs record the memory consumption of a Lambda function at each invocation. To gather insights on the efficiency of the configured memory size. These logs have to be filtered, calculated and observed over time using features such as metric math or filter, to identify actual consumption in comparison to the configured memory value. Once this data is gathered, it can then be analyzed to help decide on a more appropriate memory size configuration for the function. Although this is a valid approach, there are a few heavy liftings required before we reach the outcome.

A more efficient approach would be to use AWS Trusted Advisor check for over-provisioned memory size in AWS Lambda functions, that is backed by AWS Compute Optimizer. Compute Optimizer uses machine learning to analyze configurations and utilization metrics of your computing resources. The service identify a recommended memory size of your Lambda function, without having you to set up own baseline monitoring and analyze manually. Using this information gathered from Compute Optimizer, Trusted Advisor allows you to gain visibility on the recommended memory size, along with the estimated monthly cost savings.

Trusted Advisor supports over 20 automatic cost optimization checks that helps to discovery your resource’s detail from cost perspective, available at no cost as part of an AWS business support plan and above. The service provides checks on the following items related to Lambda function memory size:

• Over-provisioned Lambda functions memory size
• Under-provisioned Lambda functions memory size

Both over and under-provisioned memory size on a Lambda function will affect negatively to your workload. An under-provisioned memory size could lead to longer function execution time that causes performance degradation or slower response on the application. On the other hand, an over-provisioned memory size leads to unnecessary spending as Lambda function price per 1ms invocation, increases with the memory size allocation (refer to AWS Lambda pricing here).

The following image is AWS Lambda over-provisioned functions for memory size in Trusted Advisor check items, an Lambda function with 1 GB memory size allocation has been identified as over-provisioned.

Figure 5.  A screenshot of Amazon Trusted Advisor Checks for over-provisioned Lambda functions memory size

The check uses information gathered from Compute Optimizer to provide a more optimal recommended memory size which is 848 MB. The new memory size configuration provides a 6.38% Cost Savings Opportunity and $0.166 of estimated monthly savings.

Equipped with this insight in the WAFR discussion, we can immediately idenfity which memory size configuration will provide better cost efficiency, without compromizing on function performance. Accelerating our path in achieving better cost optimization in our workload.

Figure 6. Example scenario: With recommended option, the improvement could be immediately achieved

A data-driven approach can effectively produce a practical guideline for WAFR, especially in a complex workload or a large organization. In an environment with tens or even hundreds Lambda functions, it will be challenging to find out all the over-provisioned memory cases by manual process. Trusted Advisor provides a proactive way to checks to your AWS resources, that can be used to provide data and insights during WAFR conversations.

Conclusion:

AWS Well-Architected is a set of guiding design principles developed by AWS to help organizations build secure, high-performing, resilient, and efficient infrastructure of applications or workloads. Use the AWS Well-Architected Tool to review your workloads periodically to address important design considerations and ensure that they follow the best practices and guidance of the AWS Well-Architected Framework. If you need our help, you can reach out to AWS account team, or contact AWS Well-Architected Partner. For follow-up questions or comments, join our growing community on AWS re:Post.

About the Authors: