What does this AWS Solutions Implementation do?
This solution automates the steps to create a new revision for an existing AWS Data Exchange dataset in the AWS Marketplace. Instead of manually uploading and downloading dataset revisions, simply store the data you want to publish in an Amazon Simple Storage Service (Amazon S3) asset bucket and create a manifest file that references each of the files you plan to publish. When you upload the manifest file to the Amazon S3 manifest bucket, this solution initiates the process to publish a new revision. It invokes the creation of a new job in AWS Data Exchange, adds each of the files referenced in the manifest file to a new revision, then finalizes the revision once all of the assets have been added.
To retrieve the new data to their Amazon S3 bucket location as soon as the data publication completes, subscribers can use the AWS Data Exchange Subscriber Coordinator solution.
AWS Solutions Implementation overview
The diagram below presents the architecture you can automatically deploy using the solution's implementation guide and accompanying AWS CloudFormation template.
AWS Data Exchange Publisher Coordinator Solutions Implementation architecture
The AWS CloudFormation template creates an Amazon S3 bucket for submitting manifests when a dataset is ready for publication. It also deploys all of the serverless infrastructure necessary to coordinate the publication steps required for AWS Data Exchange. This includes the following:
- An Amazon S3 bucket where manifest files can be uploaded to.
- Four AWS Lambda functions: including one function that initiates the AWS Step Functions; and three functions to create a new revision of an AWS Data Exchange product, monitor job run, and finalize and publish the revision.
- An AWS Step Functions to orchestrate the management of the publishing process.
- AWS Identity and Access Management (IAM) roles that provide an AWS Step Functions and associated AWS Lambda functions with permissions to run their API calls.
Browse our library of AWS Solutions Implementations to get answers to common architectural problems.
Find AWS certified consulting and technology partners to help you get started.
Browse our portfolio of Consulting Offers to get AWS-vetted help with solution deployment.