AWS Storage Blog
Category: Intermediate (200)
Build a managed Apache Iceberg data lake using Starburst and Amazon S3 Tables
Managing large-scale data analytics across diverse data sources has long been a challenge for enterprises. Data teams often struggle with complex data lake configurations, performance bottlenecks, and the need to maintain consistent data governance while enabling broad access to analytics capabilities. Today, Starburst announces a powerful solution to these challenges by extending their Apache Iceberg […]
Build a data lake for streaming data with Amazon S3 Tables and Amazon Data Firehose
Businesses are increasingly adopting real-time data processing to stay ahead of user expectations and market changes. Industries such as retail, finance, manufacturing, and smart cities are using streaming data for everything from optimizing supply chains to detecting fraud and improving urban planning. The ability to use data as it is generated has become a critical […]
Access data in Amazon S3 Tables using PyIceberg through the AWS Glue Iceberg REST endpoint
Modern data lakes integrate with multiple engines to meet a wide range of analytics needs, from SQL querying to stream processing. A key enabler of this approach is the adoption of Apache Iceberg as the open table format for building transactional data lakes. However, as the Iceberg ecosystem expands, the growing variety of engines and languages has […]
Enhancing resource-level permission for creating an Amazon EBS volume from a snapshot
Businesses use Amazon Elastic Block Store (Amazon EBS) snapshots to capture point-in-time copies of application data volumes that can serve as baseline standards when creating new volumes. This enables them to quickly launch application workloads in different AWS Regions or meet data protection and disaster recovery requirements. Security and regulatory compliance remain top priorities as […]
Optimizing data transfers for high throughput life science instruments using AWS DataSync
Healthcare and life sciences (HCLS) customers are generating more data than ever as they integrate the use of omics data with applications in drug discovery, clinical development, molecular diagnostics, and population health. The rate and volume of data that HCLS laboratories generate are a reflection of their lab instrumentation and day-to-day lab operations. Efficiently moving […]
Understanding and monitoring latency for Amazon EBS volumes using Amazon CloudWatch
Organizations are continuing to build latency-sensitive applications for their business-critical workloads to ensure timely data processing. To make sure that their applications are working and performing as expected, users need effective monitoring and alarming across their infrastructure stack so they can quickly respond to disruptions that may impact their businesses. Storage plays a critical role […]
Enhance logs for AWS Elastic Disaster Recovery with CloudWatch Log Insights
Operational teams play a crucial role in making sure of the readiness and reliability of a disaster recovery (DR) solution. When these teams don’t have direct access to monitor the resources and services that make up a solution, it can create significant challenges. Logs provide insights into system behaviors, performance, and potential anomalies. When operations […]
Streamline search and item-level recovery with AWS Backup
UPDATE (4/29/25): Additional permissions beyond the AWS Backup default role are required to create Amazon EBS backup indexes and perform EBS file level restore. Instructions on ensuring you add the additional required permissions have been added to the post. Recovering data after a disaster or a ransomware incident headlines today’s news. But in the day-to-day, […]
Uncover new performance insights using Amazon EBS detailed performance statistics
As businesses increasingly rely on latency-sensitive applications for mission-critical workloads, the need to understand performance across the entire technology stack is essential to swiftly resolve performance bottlenecks that could affect application efficiency. Given that storage performance and stability directly impact application efficiency, reliability, scalability, and user experience, it is paramount for organizations to have the […]
Enhance resource selection in AWS Backup Policies in AWS Organizations
In today’s digital landscape, businesses rely on consistent and secure backups for data protection and disaster recovery (DR). A centralized backup policy enables organizations to enforce uniform data protection standards across departments and workloads, helping to maintain compliance and minimize risks. In the cloud, organizations use backup policies to manage data protection from a central […]