Add your own code to S3 GET requests to modify and process data as it is returned to an application
With S3 Object Lambda you can add your own code to S3 GET requests to modify and process data as it is returned to an application. For the first time, you can use custom code to modify the data returned by standard S3 GET requests to filter rows, dynamically resize images, redact confidential data, and much more. Powered by AWS Lambda functions, your code runs on infrastructure that is fully managed by AWS, eliminating the need to create and store derivative copies of your data or to run expensive proxies, all with no changes required to your applications.
S3 Object Lambda uses AWS Lambda functions to automatically process the output of a standard S3 GET request. With just a few clicks in the AWS Management Console, you can configure a Lambda function and attach it to a S3 Object Lambda Access Point. From that point forward, S3 will automatically call your Lambda function to process any data retrieved through the S3 Object Lambda Access Point, returning a transformed result back to the application. You can author and execute your own custom Lambda functions, tailoring S3 Object Lambda’s data transformation to your specific use case.
Flexibility to process data
S3 Object Lambda gives you the flexibility to invoke Lambda functions directly from S3 GET requests to process data to meet the specific requirements of your applications. S3 Object Lambda uses the fully managed infrastructure of S3 and AWS Lambda, and all of its features and capabilities. You can use new or existing AWS Lambda functions to process your data such as filtering, masking, redacting, compressing, changing the format, or modifying objects for your specific applications.
Powered by AWS Lambda functions, your code runs on infrastructure that is fully managed by AWS, eliminating the need to create and store derivative copies of your data or to run expensive proxies, all with no changes required to applications. You can run AWS Lambda functions with S3 GET requests to process objects by updating your application to use an S3 Object Lambda access point, which will invoke a Lambda function, and deliver the processed object to requesting clients.
S3 Object Lambda makes it easy to invoke Lambda functions to transform S3 data for different applications, by eliminating complex software and infrastructure, or the need to create derivative copies of your data. You can add your own code to process data as its retrieved from S3 before returning it to an application. S3 Object Lambda works with your existing applications, and you don’t need to change your code. You can easily present multiple views of your data for different applications with a standard S3 GET request.
How does S3 Object Lambda work?
S3 Object Lambda helps you to easily meet the unique data format requirements of any application without having to build and operate additional infrastructure, such as a proxy layer, or having to create and maintain multiple derivative copies of your data. S3 Object Lambda uses AWS Lambda functions to automatically process the output of a standard S3 GET request. AWS Lambda is a serverless compute service that runs customer-defined code without requiring management of underlying compute resources.
With just a few clicks in the AWS Management Console, you can configure a Lambda function and attach it to a S3 Object Lambda Access Point. Standard S3 GET requests made through an S3 Object Lambda access point will now invoke the specified Lambda function. From that point forward, S3 will automatically call your Lambda function to process any data retrieved through the S3 Object Lambda Access Point, returning a transformed result back to the application. You can author and execute your own custom Lambda functions, tailoring S3 Object Lambda’s data transformation to your specific use case. To learn more read the S3 Object Lambda user guide.
S3 Object Lambda use case: Redacting sensitive data
S3 Object Lambda can be used to redact sensitive information from an object in S3 for specific applications, without changing the existing object or creating and maintaining multiple derivative copies of the data. The Lambda function will retrieve the object from a standard S3 GET request, process the data to remove the sensitive details, and return the redacted object to the application. To learn more about other use cases, read the S3 Object Lambda user guide.
Druva delivers data protection and management across endpoints, data centers and cloud workloads eliminating dependence on complex infrastructure and time-consuming administration.
"Our 4,000 customers need to meet expanding global data protection, compliance, and security requirements. Amazon S3 Object Lambda is a market-changing innovation that brings compute resources closer to centralized data repositories, making it possible for our customers to extract more value from their data. As a platform built entirely on AWS, Druva prides itself on continuously enhancing the user experience with AWS’s latest features and technology. The combination of the Druva Cloud Platform, powered by AWS, and Amazon S3 Object Lambda will bring organizations unmatched integration opportunities as well as direct access and analysis to unlock the full potential of their backup data.”
Stephen Manley, Chief Technology Officer, Druva
Nasuni is disrupting file storage with a cloud based approached backed by Amazon S3 replacing traditional on-premises primary and secondary storage. Nasuni delivers a file storage platform using object storage delivering a simpler, lower cost, and more efficient cloud solution that scales to handle rapid unstructured data growth.
"S3 Object lambda is a game changer for Nasuni and it will be a core technology in our cloud file storage service on AWS. We use S3 Object Lambda to create a simple, scalable API to our proprietary object storage based file system. Our enterprise customers can use this to access their files directly from their S3 bucket instead of going through a Nasuni Edge Appliance. This means they can easily connect unstructured data from Nasuni to AWS analytic services like Macie and Kendra enabling enterprises to gain insights from their file data.”
Russ Kennedy, Chief Product Officer - Nasuni
PetaGene was founded in Cambridge, the birthplace of genomics, to address the rapidly growing data management problems of the genomics industry. PetaGene has become known for the industry-leading performance and usability of its genomic compression and encryption solutions — the company's software now provides encryption, selective data sharing and auditing solutions for all kinds of data, and these solutions also benefit from the same seamless transparent access methods.
"PetaGene enables organizations to protect data so that they can control and audit what different users can see, exposing only the content within each file that the user is allowed to see. PetaGene also efficiently compresses already compressed NGS genomic data up to 11x smaller without loss of any kind.
Although users can already transparently access protected and/or compressed data in their original native formats by loading our user-mode readback library into their environment, with S3 Object Lambda it is possible for users to access data in their original native formats via standard S3 GET requests, without the need for any library installation by users. This makes it even easier for customers to deploy PetaGene's data protection, compliance, and compression products for their end-users."
Dan Greenfield, PhD, Co-Founder and CEO - PetaGene
S3 Object Lambda can be set up in the S3 Management Console, with three simple steps. First, navigate to the Object Lambda Access Point tab on console. Second, create an S3 Object Lambda Access Point and in its configuration provide a name for this resource, the Lambda function to invoke against your S3 GET requests, and a supporting S3 access point. There are example Lambda function implementations in AWS documentation to help you try the service. Lastly, update your SDK and application to use the new S3 Object Lambda Access Point to retrieve data from S3 using the language SDK of your choice. S3 Object Lambda will begin to process your standard S3 GET requests. Read the user guide for more information and to try out sample S3 Object Lambda functions.