Category: Amazon Simple Storage Service (S3)

Updated 20 March: I added a phrase in the original text to mention that the AWS SDKs for Rust and Kotlin now support Amazon S3 Express One Zone. Today is AWS Pi Day! Join us live on Twitch, starting at 1 PM Pacific time. On this day 18 years ago, a West Coast retail company […]

Today we are announcing a preview of Amazon OpenSearch Service zero-ETL integration with Amazon S3, a new way to query operational logs in Amazon S3 and S3-based data lakes without needing to switch between services. You can now analyze infrequently queried data in cloud object stores and simultaneously use the operational analytics and visualization capabilities […]

Update (January 24, 2024) – The original version of this post used the s3 sync command to copy objects from my local directory to the storage bucket. The newest version of this command does not support copying to directory buckets, so I have revised the example to use s3 cp instead. — Jeff; The new […]

Update November, 16 2023 — I added a link to the documentation with the exact IAM permissions required to use this capability. — Today, we’re making available a new capability of AWS Glue Data Catalog to allow automatic compaction of transactional tables in the Apache Iceberg format. This allows you to keep your transactional data […]

I’m excited to announce today a new capability of Amazon Managed Streaming for Apache Kafka (Amazon MSK) (Amazon MSK) that allows you to continuously load data from an Apache Kafka cluster to Amazon Simple Storage Service (Amazon S3). We use Amazon Kinesis Data Firehose—an extract, transform, and load (ETL) service—to read data from a Kafka […]

The fifth annual AWS Storage Day took place on Aug. 9, 2023 and you can see the replay here. The first AWS Storage Day was hosted in 2019, and this event has grown into an innovation day that we look forward to delivering to you every year. In last year’s Storage Day post, I wrote […]

Update (September 2023) – Add information about enabling file deletion. Mountpoint for Amazon S3 is an open source file client that makes it easy for your file-aware Linux applications to connect directly to Amazon Simple Storage Service (Amazon S3) buckets. Announced earlier this year as an alpha release, it is now generally available and ready […]

Last year, Amazon S3 Glacier celebrated its tenth anniversary. Amazon S3 Glacier is the leader in cloud cold storage, and I wrote about its innovations over the last decade. The Amazon S3 Glacier storage classes provide you with long-term, secure, and durable storage options to optimally archive your data at the lowest cost. The Amazon […]