Creating customized data products on AWS Data Exchange

Companies are increasingly turning to third-party data to enhance their own data and discover new insights. AWS Data Exchange makes it easier for them to find, subscribe to, and use third-party data in the cloud. While many data subscribers are looking for data products they can immediately purchase and deploy as is, there are many scenarios in which customers need customized products to address specific needs. In November 2020, AWS Data Exchange released two features that allow data providers to customize their products: revision access rules and private products.

Revision access rules enable you to segment your historical and future revisions into different products while continuing to manage one dataset. Maybe you have a marketing customer who only needs access to the most up-to-date data or a hedge fund client who wants to backtest historical data. You can use a single dataset to create products that meet both their needs by using revision access rules.

Private products are products that are discoverable and visible only to the subscribers whom you explicitly target using their AWS account ID. Private products allow you to license data products to your customers while keeping your product, datasets, and offers confidential.

In this blog post, I will show you how to use revision access rules and private products to create custom products on AWS Data Exchange.

Prerequisites

To follow this tutorial, you must be registered as a data provider and familiar with how to create products through the AWS Data Exchange console. This workflow is covered in a previous blog post and in Providing Data Products on AWS Data Exchange in the AWS Data Exchange User Guide.

Creating a product with revision access rules

Datasets are made up of revisions. Revision access rules allow you to control which subset of those revisions are shared with your subscribers. You can add revision access rules to both public and private products by choosing revision access rules when you create a product.

During Step 3 of the product creation workflow, select the datasets to include with your product and then choose any revision access rules to apply to those datasets. Currently, you can choose to offer only historical revisions, only future revisions, all future revisions with a certain number of past revisions from a subscription start date, or a combination of these options. Subscribers get access to the data based on their subscription start date.

Example: a dataset with future revisions and one trailing revision

For a dataset with future revisions and a fixed number of trailing revisions, the subscriber gets access to the last N revisions published before their subscription and all revisions published during their subscription. N can be any integer from 1 to 100. If you create a product with only future revisions, I recommend including at least one trailing, meaning already published, revision. This ensures that subscribers start their subscription with some data available to them. To provide a dataset with future revisions and one trailing revision, do the following:

• On the Select revision access rules page, choose A fixed number of training revisions published prior to subscription.
• In the Included number of training published revisions box, enter 1.
• Check the radio button next to All future revisions published during subscription duration. Refer to the following screenshot, which shows the Select revision access rules page with these selections.

Example: a dataset with all trailing revisions

To provide a dataset with all pre-existing or trailing revisions published prior to subscription but no future revisions, do the following.

• On the Select revision access rules page, select All pre-existing revisions published prior to subscription.
• Select No future revisions. Refer to the following screenshot, which shows the Select revision access rules page with these selections.

• Once you’ve chosen your revision access rules, at the bottom of the page select Next.
• Enter offer details, like duration and price for your product. Choose Next.
• Review your product description, revisions access rules, and offer details. To make your product available to subscribers, select Publish.

Viewing revision access rules

Subscribers can see which revision access rules are in place for a product. To do this, navigate to the AWS Data Exchange catalog. At the top of the product’s detail page, choose the Data sets tab. The following screenshot shows a dataset that includes the last 100 revisions and all future revisions.

Creating a private product

Now that you understand how to create custom products using revision access rules, I will show you how to create private products. You can use private products on AWS Data Exchange to deliver custom-made data or data not intended for the general public.

1. On the AWS Data Exchange console left navigation, choose Products. Then in the top right, choose Publish new product. In Step 1: Product visibility, select Private and then choose the sensitive information category that the dataset in this product falls under. Choose Next. For more information, see Sensitive Categories of Information in the AWS Data Exchange User Guide.
2. On the next page, Step 2: Define product, enter details about your product and choose Next. For Step 3: Add data, choose which datasets to add and any revision access rules to apply to this product, and choose Next.
3. Step 4: Add custom offer is where you target your private product and private offer to a specific account. In the Offer types section, choose Private offer. In the Subscriber account information section, you can target up to 25 accounts with a single offer by entering a subscriber’s account ID and choosing Add. For testing purposes, target a $0 offer at your own account, which enables you to see what a subscriber would see when the offer is targeted toward them.
4. On the Add custom offer page, complete the rest of the offer detail sections, including offer expiration date, refund policy, and auto-renewal settings. Choose Next. Review the final product. To publish your private product and send the private offer to the account or accounts that you targeted, choose Publish.

Congratulations! You have created a private product with a targeted private offer and specific revision access rules. The accounts that you targeted in the private offer can view and subscribe to your product within a few hours after publishing.

Adding private offers to your private product

After you have created a private product, you can add additional private offers to it. To do that, in the AWS Data Exchange console Products tab, choose the product to modify, choose Private offers, and choose Create.

Expiring a private offer and unpublishing a data product

You can manually expire an offer at any time before the offer expiration date you previously selected. Expired offers are still visible to subscribers and won’t cancel any subscriptions that were created before the offer was expired. To expire a private offer, do the following:

1. On the Products tab, choose the product that includes the offer to modify.
2. Choose Private offers, and choose the offer to expire.
3. On the offer detail page top right, choose Expire.

Unpublishing a data product

If you made an error or need to unpublish a data product, you can do so by going to the AWS Data Exchange console, navigating to the product’s Product overview page, and choosing Unpublish.

All public and private unpublished products that still have active subscriptions are available to existing subscribers until the end of the subscription agreement. Customers that haven’t subscribed to the unpublished product are unable able to see it in the catalog, but customers with an active subscription are able to view the product. Active subscribers can’t renew their subscription at the end of their agreement, but they can receive new revisions published to that product.

Conclusion

AWS Data Exchange enables you to customize your data products and deliver them quickly and securely to your customers. In this post, I showed how to create products using revision access rules. Revision access rules enable you to separate access to historical and future revisions into different product offerings. I also showed you how to create private products that are targeted to specific customer accounts. To learn more about the service, see AWS Data Exchange.

About the author

Anna Whitehouse is a product manager with AWS Data Exchange based out of New York, NY. Before joining AWS, Anna spent five years in financial services working with customers to solve business problems using third-party data.