AWS Storage Blog

Tag: AWS Lambda

Amazon S3 Batch Operations featured image

Updating Amazon S3 object ACLs at scale with S3 Batch Operations

Access control lists (ACLs) are permission sets associated with data or other system resources that dictate access permissions, and they have been a staple of data security for decades. You may come across a situation where you want to update the ACL on a large number of files, perhaps billions or more. For example, you […]

Read More
AWS DataSync Featured Image 2020

How to move and store your genomics sequencing data with AWS DataSync

Genomics data is expanding at a rate exceeding Moore’s law according to the National Human Genome Research Institute. As more sequencing data is produced and researchers move from genotyping to whole genome sequencing, the amount of data produced is outpacing on-premises capacity. Organizations need cloud solutions that help manage data movement, storage, and analysis. The […]

Read More
S3 Security

Reduce encryption costs by using Amazon S3 Bucket Keys on existing objects

As more organizations look to operate faster and at scale, they need ways to meet critical compliance requirements and improve data security. Encryption is a critical component of a defense in depth strategy, and when used correctly, can provide an additional layer of protection above basic access control. However, workloads that access millions or billions […]

Read More
Amazon EFS serverless featured image

Analytical processing of millions of cell images using Amazon EFS and Amazon S3

Analytical workloads such as batch processing, high performance computing, or machine learning inference often have high IOPS and low latency requirements but operate at irregular intervals on subsets of large datasets. Typically, data is manually copied between storage tiers in preparation of processing, which can be cumbersome and error-prone. Given this, IT teams want to […]

Read More
Amazon S3 Object Lambda

How Trend Micro uses Amazon S3 Object Lambda to help keep sensitive data secure

Does your application handle data that is uploaded by hundreds of thousands of end users? Is that same underlying data then shared across the same magnitude of users? Being able to scan data for malware before it’s returned to an application helps keep sensitive data secure, provides protection regardless of when the data was initially […]

Read More

How SkyWatch built its satellite imagery solution using AWS Lambda and Amazon EFS

SkyWatch is on a mission to democratize remote sensing data through a simple user experience. Every day, trillions of pixels of Earth observation imagery are captured by satellites orbiting our planet. New applications for this data are developed every week, with demand increasing across many industries. Examples include commercial applications, such as construction, finance, and […]

Read More
Amazon S3 Glacier Storage Classes

Modernizing NASCAR’s multi-PB media archive at speed with AWS Storage

The National Association for Stock Car Auto Racing (NASCAR) is the sanctioning body for the No. 1 form of motor sports in the United States, and owns 15 of the nation’s major motorsports entertainment facilities. About 15 years ago NASCAR began to collect all the video, audio, and image assets from over the last 70+ […]

Read More
AWS Transfer Family Featured Image

Enabling user self-service key management with AWS Transfer Family and AWS Lambda

Customers who use the AWS Transfer Family service are typically exchanging files with their business partners who provide them with SSH public keys. In a large-scale deployment of the AWS Transfer Family service, public key management eventually becomes a time-consuming task to refresh expired keys and rotate keys for security. When using custom identity providers (custom IdP), […]

Read More
Amazon S3 Object Lambda

Automate the configuration of Amazon S3 Object Lambda to process data as its retrieved

Customers often have multiple applications consuming information from a single dataset stored in Amazon S3, but each application requires different variations to meet their specific needs. For example, one application may require that data be in a redacted CSV format, while another application requires unredacted information. As another example, a stock image website may require […]

Read More
CloudEndure-AWS-Storage-Blog

Enhanced disaster recovery monitoring with CloudEndure and Amazon CloudWatch

Update (March 4, 2022): Updated Amazon CloudWatch events section to utilize Amazon EventBridge rules instead, allowing you to further customize your serverless event architecture. AWS is also deprecating the requests module in botocore to improve flexibility and performance, and added the ability to use Layers within AWS Lambda to continue using this module. Code has […]

Read More