Access your Amazon S3 Storage Lens metrics in AWS Partner applications

Managing cloud storage at scale requires the right metrics and tools to maintain visibility into your storage footprint, and uncover opportunities to reduce storage costs or apply data protection best practices.

With the launch of S3 Storage Lens in November 2020, customers gained access to the first cloud storage analytics solution offering organization-wide visibility into object storage, with point-in-time metrics, detailed visualizations, and granular drill-downs by storage class, Region, and bucket. S3 Storage Lens includes 29 usage and activity metrics for maintaining detailed visibility into your storage, such as incomplete multipart upload bytes, encrypted object bytes, and noncurrent version object counts.

In the last year, we’ve seen rapid adoption of S3 Storage Lens among customers across multiple industries and segments, and we’ve received positive feedback on the usability of the Storage Lens dashboard, which is built in to the Amazon S3 console and provides flexible filtering and drill-downs to analyze with ease. While many customers appreciate having convenient access to storage insights in the same interface where they configure their storage, other customers voiced preferences to monitor their storage centrally alongside other AWS applications, leading to the launch of the Amazon CloudWatch publishing option for S3 Storage Lens. With S3 Storage Lens metrics in CloudWatch, you can create a unified view of operational health for your AWS resources, applications, and services. You can also use CloudWatch to detect anomalous behavior, set alarms, take automated actions, troubleshoot issues, and discover insights to help you implement S3 best practices.

Customers also tell us they want access to their S3 Storage Lens metrics via their preferred third-party monitoring and observability tools to take advantage of specialized features and unique data visualizations. Based on this feedback we’ve made it possible for any monitoring and observability vendor to access S3 Storage Lens metrics, and we’ve already worked with several of the leading AWS Partners in this space to create pre-integrated capabilities. Read on for more details on how this works, and to see a list of the partners.

How it works

S3 Storage Lens now includes an option to publish metrics to Amazon CloudWatch on a daily basis. Once metrics are published to CloudWatch, they become available for consumption via the CloudWatch Get Metrics APIs. As such, AWS Partners who update their CloudWatch API integration can programmatically access S3 Storage Lens metrics on your behalf to surface them in their specialized monitoring features and unique data visualizations.

To get started, first navigate to the S3 Storage Lens dashboard configuration page. There you can select the CloudWatch publishing option, under Advanced metrics and recommendations (as shown in the following screenshot).

S3 Storage Lens advanced metrics and recommendations includes CloudWatch metrics publishing for no additional charge (however, CloudWatch API charges still apply). For S3 Storage Lens advanced metrics and recommendations pricing, visit the Amazon S3 pricing page.

After enabling this option, your S3 Storage Lens metrics will be ready to be accessed by your AWS Partner via the CloudWatch API within 24 hours, and can be found under the S3 Storage Lens metrics namespace.

Note: S3 Storage Lens metrics are produced daily and are not compatible with CloudWatch metrics streams. AWS Partners relying on metrics currently do not have access to S3 Storage Lens metrics.

Pre-integrated AWS Partners

Here are some of the AWS Partners where you can already access S3 Storage Lens metrics. You can get in touch with any of these partners via the AWS Marketplace, and you can also contact other AWS Partners to inquire about the status of their S3 Storage Lens integration.

Datadog

Datadog is the SaaS monitoring and security platform for cloud applications that integrates and automates infrastructure monitoring, application performance monitoring, security monitoring and log management to provide unified, real-time observability into an organization’s entire technology stack. Datadog’s integration with Amazon S3 Storage Lens makes it easy for customers to understand their S3 storage usage and activity alongside their other infrastructure and applications.

Elastic

Elastic Cloud on AWS is brought to you by Elastic, the creators of Elasticsearch, Logstash, Kibana, and Beats. Elastic Cloud on AWS gives users the ability to run pre-built SaaS solutions for Enterprise Search, Security, and Observability, including logging, application performance management, metrics. Elastic Observability provides access to S3 Storage Lens metrics to help you get crystal clear insight into your storage usage.

Moogsoft

Moogsoft uses patented AI to automatically surface incidents by correlating S3 Storage Lens metrics together with metrics from over 80 AWS services and data from other monitoring systems and tools. This automated anomaly detection and correlation reduces manual repetitive work (Toil) for SREs, DevOps and IT Op and enables them to diagnose and fix issues much faster. Find S3 Storage Lens as one of the Ingestion Services in the Moogsoft platform.

PagerDuty

PagerDuty Operations Cloud™ empowers the right action when seconds matter. With 600+ native integrations and the ability to build and customize workflows with the extensible PagerDuty APIs, we integrate data from all your tools to give you insights into your IT infrastructure. PagerDuty is helping modern enterprises redefine their approach to operations in a digital-first world. Customers can use PagerDuty to create alerts for S3 Storage Lens metrics.

Availability and pricing

Amazon S3 Storage Lens is available in all AWS Regions, as well as the AWS China (Beijing) Region, operated by Sinnet, and the AWS China (Ningxia) Region, operated by NWCD. S3 Storage Lens advanced metrics and recommendations includes support for CloudWatch metrics publishing for no additional charge, however separate CloudWatch API charges apply. For S3 Storage Lens advanced metrics and recommendations pricing, visit the Amazon S3 pricing page.

Refer to the documentation to learn how to turn on CloudWatch publishing in S3 Storage Lens with just a few clicks of the S3 console, or using Amazon S3 API or CLI. To learn more about Amazon CloudWatch APIs, refer to our website.

Conclusion

In this blog post, we’ve detailed how you can extend your storage observability by enabling S3 Storage Lens metrics to appear in one of the several leading third-party applications in the monitoring and observability space. In doing so, you can unlock specialized capabilities and unique visualizations within these partner applications for use with S3 Storage Lens metrics. This gives you more options for maintaining detailed visibility of your storage usage and activity, and helps you continually apply best practices in managing your S3 storage footprint. Contact these and other AWS Partners in the AWS Marketplace for more details on using S3 Storage Lens metrics within their tools.

Thanks for reading this blog post and learning more about using S3 Storage Lens metrics within several AWS Partner applications through the Amazon CloudWatch API. If you have any comments or questions, feel free to leave them in the comments section.