Skip to main content

AWS Solutions Library

Guidance for Deploying a Data Transfer Dashboard for AdTech on AWS

Go to sample code

Overview

This Guidance shows how to use AWS Cost and Usage Reports (AWS CUR) and Amazon Athena to build a dashboard that gives you a centralized view of your data transfer costs and usage. In advertising and marketing technology, it’s important to understand data transfer costs and trends. However, this can be difficult if you don’t have a quick way to analyze your data transfer object bill. With this interactive Amazon QuickSight dashboard, your organization can investigate specific data-transfer-related operations from your linked accounts and track your costs to inform your business decisions.

How it works

These technical details feature an architecture diagram to illustrate how to effectively use this solution. The architecture diagram shows the key components and their interactions, providing an overview of the architecture's structure and functionality step-by-step.

Download the architecture diagram

Deploy with confidence

Ready to deploy? Review the sample code on GitHub for detailed deployment instructions to deploy as-is or customize to fit your needs. 

Go to sample code

Well-Architected Pillars

The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible

Amazon CloudWatch stores logs of successful and failed partition logs daily in an S3 bucket for use by AWS CUR . You can use these logs to identify any failures for specific days and correct the issue by adding missing partitions. CloudWatch logs also provide information useful for querying and visualizing daily data transfer data in addition to enabling automated alerts and monitoring capabilities.

Read the Operational Excellence whitepaper 

This Guidance relies on your preexisting use of AWS CUR . The integration with Amazon S3 uses the default S3 bucket policy for new or existing buckets so that the AWS CUR account owner will become the authorized bucket owner and have access to read the AWS CUR data.

Read the Security whitepaper 

This Guidance relies on serverless services ( Athena , Amazon S3 , AWS Glue , QuickSight , and CloudWatch ) so that you don’t have to provision compute or manage any resources. These services are resilient and scale based on demand. Amazon S3 provides reliable storage by storing copies of objects in multiple Availability Zones. Athena views, AWS Glue databases and schema, and QuickSight are templatized, and you can use AWS CloudFormation to deploy them.

Read the Reliability whitepaper 

Amazon S3 daily partitions, which are compressed AWS CUR files in Apache Parquet format, enable better performance when you query them from Athena . Additionally, QuickSight supports SPICE (Super-fast, Parallel, In-memory Calculation Engine) storage of the query result, accelerating the data transfer analysis and visualization for your organization and linked accounts.

Read the Performance Efficiency whitepaper 

Athena , Amazon S3 , AWS Glue , and QuickSight use a pay-as-you-go model, meaning you pay only for the resources you consume. Additionally, because these services are serverless, they will scale as needed, so you don’t need to provision resources to run 24/7. This helps you optimize the cost of visualizing your data usage.

Read the Cost Optimization whitepaper 

Athena compresses AWS CUR data in S3 buckets and uses the Apache Parquet format, which significantly reduces the object size in storage, lowering your storage footprint. This Guidance only creates the minimum resources (such as Athena views that are created with AWS CUR schema) required to query data at runtime. Additionally, you can set up Amazon S3 lifecycle policies to delete objects that are no longer needed for analyzing your data transfer usage. Finally, this Guidance uses serverless, scalable compute that does not run when idle, optimizing your energy use.

Read the Sustainability whitepaper 

Disclaimer

The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.