AWS Storage Blog
Category: Amazon CloudWatch
How Visual Layer builds high quality datasets on Amazon S3
Companies from different industries use data to help their Artificial Intelligence (AI) and Machine Learning (ML) systems make intelligent decisions. For ML systems to work well, it is crucial to make sure that the massive datasets used for training ML models are of the highest quality, minimizing noise that can contribute to less-than-optimal performance. Processing […]
Configuring the auto-expansion of Amazon FSx for OpenZFS with Amazon CloudWatch and AWS Lambda
Today’s demanding workloads such as database, rendering farm, analytics and ML workloads, have increasingly demanding IO requirements. These workloads need a reliable storage infrastructure that provides sufficient storage capacity, IOPS, and throughput. As customers move more workloads to the cloud, they want to benefit from the agility and performance capabilities of the cloud as their […]
Encrypt and decrypt files with PGP and AWS Transfer Family
1/11/2024: Updates made due to CloudShell migration to Amazon Linux 2023 (AL2023). Protecting sensitive data is not a novel idea. Customers in industries like financial services and healthcare regularly exchange files containing sensitive data, including Personal Identifiable Information (PII) and financial records with their users. Pretty Good Privacy (PGP) encryption of these files is often […]
Performance analysis for different Amazon EFS throughput modes via Amazon CloudWatch
When I talk with customers about their file storage, I frequently get asked “How can I determine the right throughput capacity for my file storage?” The simple answer is through monitoring the performance of your workload to determine the right performance configuration for your file storage. Throughput modes for Amazon Elastic File System (EFS) determine […]
Maximizing price performance for big data workloads using Amazon EBS
Since the emergence of big data over a decade ago, Hadoop – an open-source framework that is used to efficiently store and process large datasets – has been crucial in storing, analyzing, and reducing that data to provide value for enterprises. Hadoop lets you store structured, partially structured, or unstructured data of any kind across […]
Improve application resiliency with Amazon EBS volume metrics and AWS Fault Injection Simulator
When business build applications, it is crucial to monitor those applications and take measures if and when needed to avoid downtime, and potentially, revenue loss. Choosing the correct metrics to monitor and setting up alarms as needed is table stakes for customers to achieve their application resiliency and availability goals. Amazon Elastic Block Store (Amazon […]
Automating Amazon CloudWatch dashboard creation for Amazon EBS volume KPIs
Enterprises can benefit significantly from optimizing block storage performance in the cloud. One of the primary benefits is having faster and more reliable data access to support critical business operations and real-time applications by achieving reduced latency and higher throughput. Another benefit is realizing cost savings by increasing storage operational efficiency to reduce the need […]
Retrieving previous versions of S3 bucket policies
When dealing with applications or storage, it is often helpful to log previous policy configurations and have the ability to retrieve them. This can help you managed and troubleshoot configuration changes and comply with security regulations. Amazon Simple Storage Service (Amazon S3) bucket policies are resource-based policies that you can use to grant access permissions […]
Modern data protection architecture on Amazon S3: Part 2
Update (12/11/2023): As of November 20, 2023, Amazon S3 supports enabling S3 Object Lock on existing buckets. Keeping data secure and usable in unforeseen circumstances like accidental breaches, human error, and hacking is critical to business continuity and success. To effectively mitigate the impact of these events on business-critical assets, one of the recommended strategies […]
Modern data protection architecture on Amazon S3: Part 1
Update (12/11/2023): As of November 20, 2023, Amazon S3 supports enabling S3 Object Lock on existing buckets. Keeping data secure and usable in unforeseen circumstances like accidental breaches, human error, and hacking is critical to business continuity and success. To effectively mitigate the impact of these events on business-critical assets, one of the recommended strategies […]