AWS Storage Blog
Tag: AWS Lambda
Creating an ETL pipeline trigger for existing AWS DataSync tasks
Organizations look for ways to leverage the compute power of the cloud to analyze their data and produce reports to help drive business decisions. They want to load their data sets into extract-transform-load (ETL) pipelines for data processing. Once the data is processed, business decision makers at these organizations rely on accurate report generation to […]
Preserving last-modified timestamps when restoring Amazon S3 objects with AWS Backup
Customers operating in highly regulated industries are usually subject to rules mandating that data integrity be maintained and available throughout its entire lifetime. To meet integrity requirements, data must be restorable along with any associated audit trail and metadata information, such as object creation dates, last modified timestamps, and tags. When restoring backups of Amazon […]
How Audible uses Amazon S3 Object Lambda to improve streaming playback performance
Audible is a leading creator and provider of premium audio storytelling that offers its customers a new way to enhance and enrich their lives daily. Audible content includes over 790,000 audiobooks, podcasts, and Audible Originals. Audible has millions of members worldwide, who subscribe to one of 10 localized services designed for customers in Australia, Canada, France, […]
Retrieving previous versions of S3 bucket policies
When dealing with applications or storage, it is often helpful to log previous policy configurations and have the ability to retrieve them. This can help you managed and troubleshoot configuration changes and comply with security regulations. Amazon Simple Storage Service (Amazon S3) bucket policies are resource-based policies that you can use to grant access permissions […]
Data migration and cost saving at scale with Amazon S3 File Gateway
Migrating data to the cloud requires experience with different data types and the ability to preserve source data structure and metadata attributes. Customers often have on-premises file data stored on traditional file servers, retaining original timestamp of data creation for varying reasons including data lifecycle management. Customers find it challenging to identify a path to […]
Automatically compress and archive satellite imagery for Amazon S3
Satellite imagery often comes as large, high-resolution files, and organizations that work with this data typically have high storage costs. Additionally, large imagery files can take time and resources when downloaded for use with machine learning (ML), data analytics tools, or manual analyst review. Using standard compression techniques lets us achieve reductions in file size […]
Modern data protection architecture on Amazon S3: Part 2
Keeping data secure and usable in unforeseen circumstances like accidental breaches, human error, and hacking is critical to business continuity and success. To effectively mitigate the impact of these events on business-critical assets, one of the recommended strategies is creating immutable, unchangeable copies of those assets and storing them in isolated, secondary accounts with restricted […]
Modern data protection architecture on Amazon S3: Part 1
Keeping data secure and usable in unforeseen circumstances like accidental breaches, human error, and hacking is critical to business continuity and success. To effectively mitigate the impact of these events on business-critical assets, one of the recommended strategies is creating immutable, unchangeable copies of those assets and storing them in isolated, secondary accounts with restricted […]
Automating AWS Backup pre- and post-script execution with AWS Step Functions
Customers execute custom scripts before or after a backup job to automate and orchestrate required and repetitive tasks. For example, customers running applications hosted in Amazon Elastic Compute Cloud (EC2) instances use scripts to complete application transactions, flush the buffers and caches, stop file I/O operations, or ensure that the application is idle, bringing the […]
Troubleshooting automated pre- and post-scripts for AWS Backup
Customers can use event-driven architectures with decoupled tasks to automate and orchestrate custom scripts for backup jobs. With event-driven architectures, troubleshooting is key to understanding failures at the component levels in order to resolve issues that arise and keep the entire automated workflow running smoothly. In the first post in this two-part blog series, we […]