AWS Storage Blog
Category: Amazon S3 Glacier Deep Archive
How London Stock Exchange Group migrated 30 PB of market data using AWS DataSync
London Stock Exchange Group (LSEG) has 30 PB of Tick History-PCAP data, which is ultra-high-quality global market data that is based on raw exchange data, timestamped to the nanosecond. An additional 60 TB is generated every day. LSEG sought to migrate their data from Wasabi cloud storage, LSEG was looking for a new solution to […]
Automating retrievals from the Amazon S3 Glacier storage classes
Faced with increasing amounts of data and a tightening economic climate, enterprises are looking to save money on their storage costs by moving rarely needed data to archival storage options. The least costly options require your internal systems to support receiving data back in hours or days, often called asynchronous retrievals. With this time delay, […]
Identify cold objects for archiving to Amazon S3 Glacier storage classes
Update (02/13/2024): Consider Amazon S3 Lifecycle transition fees that are charged based on the total number of objects being transitioned, the destination storage class (listed on the Amazon S3 pricing page), as well as the additional metadata charges applied. You can use the S3 pricing calculator to estimate the total upfront and monthly costs by […]
Reduce recovery time and optimize storage costs with faster restores from Amazon S3 Glacier storage classes and Commvault
Data is the lifeblood of any modern business. Organizations are storing more copies of their application data than ever before to recover from data loss, repair data corruption or ransomware damage, respond to compliance requests, and become more data driven. Storing more data at reduced cost enables businesses to extract more value and insights to […]
Automatically compress and archive satellite imagery for Amazon S3
Satellite imagery often comes as large, high-resolution files, and organizations that work with this data typically have high storage costs. Additionally, large imagery files can take time and resources when downloaded for use with machine learning (ML), data analytics tools, or manual analyst review. Using standard compression techniques lets us achieve reductions in file size […]
Restoring archived objects at scale from the Amazon S3 Glacier storage classes
Update (7/26/2024): You no longer need to optimize the S3 Inventory report using Amazon Athena. Amazon S3 will automatically optimize your S3 Batch Operations restore job to achieve the fastest retrieval throughput. For more guidance on using batch operations, learn more in the S3 User Guide. Every organization around the world has archival data. There is […]
How Indus OS cost-effectively transitioned billions of small objects between Amazon S3 storage classes
Indus OS is a mobile platform that enables content and application discovery for users, application developers, and original equipment manufacturers (OEMs). Powered by artificial intelligence, Indus App Bazaar curates locally relevant apps and content for users based on their demographics, lingual preferences, and behaviour, and offers an intuitive user interface with content-led discovery. Indus App […]
Automatically archive and restore data with Amazon S3 Intelligent-Tiering
Customers of all sizes, in all industries, are using data lakes to transform data from a cost that must be managed, to a business asset. From time to time, data scientists and business analysts need to restore subsets of historical datasets for longitudinal studies, machine learning retraining, and more. However, users commonly write queries that don’t […]
Restore data from Amazon S3 Glacier storage classes starting with partial object keys
When managing data storage, it is important to optimize for cost by storing data in the most cost-effective manner based on how often data is used or accessed. For many enterprises, this means using some form of cold storage or archiving for data that is less frequently accessed or used while keeping more frequently used […]
How to move and store your genomics sequencing data with AWS DataSync
Genomics data is expanding at a rate exceeding Moore’s law according to the National Human Genome Research Institute. As more sequencing data is produced and researchers move from genotyping to whole genome sequencing, the amount of data produced is outpacing on-premises capacity. Organizations need cloud solutions that help manage data movement, storage, and analysis. The […]