Visualize and gain insights into your AWS cost and usage with Cloud Intelligence Dashboards and CUDOS using Amazon QuickSight

Across all industry segments, our customers require better visibility into their AWS usage to help them understand the return on their investment, achieve operational efficiency, and make business decisions that have financial impact. As organizations mature, customers need to answer simple but granular operational questions related to:

• Which key areas should I focus to optimize my AWS usage and costs?
• How do I track the impact of my cost optimization on my business?
• How do I gain resource level granularity of the AWS services my workloads use?

You have access to raw cost and usage data through the AWS Cost and Usage (AWS CUR) reports. These reports contain the most comprehensive information about your AWS usage and costs. Financial teams need this data so they have an overview of their monthly, quarterly, and yearly AWS spend, but this data is equally valuable for technical teams who need detailed resource-level granularity to understand which resources are contributing to the spend. AWS Cost Explorer provides a high-level view of costs and usage, using the same dataset that is used to generate the AWS Cost and Usage Reports. To extract resource-level granularity, you can use Amazon Athena queries, which requires familiarity and previous experience to build complex SQL queries.

Cloud Intelligence Dashboards for cost optimization

Having dashboards that provide prebuilt visualizations can help you get a detailed view of your AWS usage and costs. They can help you identify which service and which underlying resources contribute the most toward your AWS spend.

The Cloud Intelligence Dashboards are a collection of Amazon QuickSight dashboards. They offer powerful visuals, in-depth insights, and intuitive querying without having to build complex solutions or share your cost data with third-party companies. The Cost Intelligence Dashboard, CUDOS Dashboard, Trusted Advisor Organization (TAO) Dashboard, and Trends Dashboard are built on native AWS services. They are inherently secure because the data resides in the organization. They inherit all the features of Amazon QuickSight, including integration with AWS Identity and Access Management, which makes them highly secure, and Amazon QuickSight being a serverless service allows you to pay as you go and scale on demand.

You do not need coding or SQL skills to customize these dashboards. The visualizations include Machine Learning (ML) driven insights, live trends, actionable recommendations, links to relevant blog posts and AWS service documentation that help you make informed business decisions.

In the words of John Richardson, the head of Cloud Services at YouView TV, says, “The CUDOS Dashboard allows our stakeholders to quickly visualize where we are in terms of our cost management strategy at any time, with the ability to delve deeper into the detail if required. The use of the dashboard along with other tools is helping us to set better budget targets and actively manage them.”

In this blog post, we will cover the CUDOS Dashboard in detail.

CUDOS dashboard demo

To familiarize yourself, try the CUDOS dashboard demo. You can use the Select Dashboard drop-down, as shown in Figure 1:

Figure 1: Dashboard demo

FinOps teams, DevOps teams, engineering leads, and other teams will find the information in this dashboard useful. This dashboard is a one-time deployment and can support multi-payer organizational setup. AWS CUR is the main data source for this dashboard.

CUDOS dashboard architecture overview

Start by creating a CUR report with Amazon Athena integration enabled, in the Billing and Cost management console. This will ensure the CUR files are uploaded to an Amazon Simple Storage Service (Amazon S3) bucket on a regular basis. For multi-payer account setup, we recommend that you use S3 replication to create a copy of the CUR in an S3 bucket in your cost optimization account (Option 2 in Configure Cost and Usage reports, in AWS Account Setup Lab).

The Amazon Athena integration with AWS CUR provides AWS CloudFormation template in the S3 bucket configured for the CUR. Upon deploying this AWS CloudFormation template, an AWS Glue crawler is set up to index the S3 bucket for data, using AWS Glue Data Catalog. This data is then made available through tables in Amazon Athena. The CUDOS workshop provides a series of SQL queries when ran in Athena will create views. These views are further used to create datasets in Amazon Quicksight. You can deploy an Amazon QuickSight dashboard using a shared template that consists of sample visualizations and insights which provides immediate value to the viewer. After the dashboard is deployed, these visuals are refreshed on regular basis using the QuickSight SPICE refresh feature.

The CUDOS Dashboard provides operational metrics such as Single-AZ Amazon RDS instances and inter-AZ data transfer generators. It also provides optimization and guidance for end user compute services like Amazon WorkSpaces and Reserved Instance savings for Amazon DynamoDB. Visuals with granular time intervals make it possible to view AWS Lambda usage and costs and the hourly usage and costs of your AWS Fargate cluster.

Figure 2: CUDOS Dashboard Architecture Overview

Dashboard deployment steps

To deploy the CUDOS Dashboard, check the manual deployment option in the AWS Well-Architected Cost Optimization lab. This deployment process offers you a look at how the CUDOS Dashboard operates under the hood. Once the prerequisites are completed, it should take approximately 60-90 minutes to complete the lab. If you prefer an automated deployment process, it’s available in CUDOS GitHub repo.

Customization ideas

After you deploy the dashboard, you can save it as a new analyses and customize it according to your needs. You can modify existing visuals by applying filters and create new visuals to focus on a particular service or resource during specific timeframe. You can also add more data sources. For example, AWS Organizations data can be used to enhance dashboards with a wider range of data, not just CUR data. As an author of this analyses, you can publish and share this modified dashboard with other readers in the Amazon QuickSight account. For more information, check Working with Amazon QuickSight Visuals in the Amazon QuickSight User Guide and the Amazon QuickSight YouTube channel.

Conclusion

In this blog post, we introduced you to Cloud Intelligence Dashboards and unveiled how the CUDOS Dashboard can help you visualize cost and usage data from your CUR reports. We’ll follow this post with other posts about the Cost Intelligence Dashboard and the Trusted Advisor Organization (TAO) Dashboard, so stay tuned!

About the authors