Skip to main content

AWS Solutions Library

Guidance for Object-Level Insights and Cost Savings with Amazon S3

Go to sample code

Overview

This Guidance shows how you can gain granular insights into the access patterns of objects stored in Amazon Simple Storage Service (Amazon S3) and use those insights to optimize storage costs and energy usage. It helps you identify objects that have not been accessed for a specified period and transition those objects to cheaper storage classes, realizing cost savings by storing infrequently accessed data in more affordable tiers. Additionally, you can configure rules to automatically delete objects that have not been accessed within a set time frame, helping to optimize storage by removing data that is no longer needed. These new capabilities provide better visibility and control over Amazon S3 object lifecycle management, so you can reduce storage costs and energy expenditures by aligning your storage strategies with your object access patterns.

How it works

These technical details feature an architecture diagram to illustrate how to effectively use this solution. The architecture diagram shows the key components and their interactions, providing an overview of the architecture's structure and functionality step-by-step.

Download the architecture diagram

Deploy with confidence

Ready to deploy? Review the sample code on GitHub for detailed deployment instructions to deploy as-is or customize to fit your needs.

Go to sample code

Well-Architected Pillars

The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.

In this Guidance, Amazon S3, Athena, QuickSight, and AWS Glue work together to help you reduce your storage costs and identify object-level patterns happening in your Amazon S3 buckets. These insights and visualizations, which include metrics on daily bucket operations, can help you identify candidates for transition or deletion to promote operational excellence.

Read the Operational Excellence whitepaper

Amazon S3 buckets block public access and are encrypted by default. Additionally, Athena encrypts data while saving query results and while communicating to Amazon S3 buckets. QuickSight supports encryption for all data transfers. Finally, you can use AWS Identity and Access Management (IAM) access-control roles and policies to manage your resources and protect data.

Read the Security whitepaper

All the services used in this Guidance are available in most AWS Regions. For example, QuickSight is available in 21 Regions worldwide. Each Region is a separate geographic area designed to be isolated from the other Regions and contains multiple isolated locations known as Availability Zones. This design maximizes fault tolerance and stability. Additionally, Amazon S3 is designed for 99.999999999 (11 nines) percent durability.

Read the Reliability whitepaper

This Guidance uses managed and serverless services like Athena, QuickSight, and AWS Glue, so you don’t have to manage resources and the underlying infrastructure. Moreover, this Guidance will scale up automatically to provide analytics on Amazon S3 buckets. By using these services together, you can also gain the insights needed to optimize the performance of your Amazon S3 buckets.

Read the Performance Efficiency whitepaper

By using Athena, QuickSight, and AWS Glue to generate insights on your Amazon S3 bucket use, you can make more informed object-storage decisions, resulting in cost savings. For example, using the insights you gain, you can categorize objects that have been colder over a certain period of time and choose to transition them to a cheaper storage class or expire them altogether.

Read the Cost Optimization whitepaper

This Guidance only uses serverless services (including Athena, QuickSight, and AWS Glue), so they scale up and down based on load, and you don’t have to provision or manage any hardware. As a result, you don’t waste energy through overprovisioning resources, and you don’t have to manage any of your underlying infrastructure.

Read the Sustainability whitepaper

Disclaimer

The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.