Enhancing recommendation filters by filtering on item metadata with Amazon Personalize

This blog post was last reviewed or updated April, 2022 with database schema updates.

We’re pleased to announce enhancements to recommendation filters in Amazon Personalize, which provide you greater control on recommendations your users receive by allowing you to exclude or include items to recommend based on criteria that you define. For example, when recommending products for your e-retail store, you can exclude unavailable items from recommendations. If you’re recommending videos to users, you can choose to only recommend premium content if the user is in a particular subscription tier. You typically address this by writing custom code to implement their business rules, but you can now save time and streamline your architectures by using recommendation filters in Amazon Personalize.

Based on over 20 years of personalization experience, Amazon Personalize enables you to improve customer engagement by powering personalized product and content recommendations and targeted marketing promotions. Amazon Personalize uses machine learning (ML) to create high-quality recommendations for your websites and applications. You can get started without any prior ML experience using simple APIs to easily build sophisticated personalization capabilities in just a few clicks. Amazon Personalize processes and examines your data, identifies what is meaningful, automatically picks the right ML algorithm, and trains and optimizes a custom model based on your data. All of your data is encrypted to be private and secure, and is only used to create recommendations for your users.

Setting up and using recommendation filters is simple, taking only a few minutes to define and deploy your custom business rules with a real-time campaign. You can use the Amazon Personalize console or API to create a filter with your business logic using the Amazon Personalize domain specific language (DSL). You can apply this filter while querying for real-time recommendations using the GetRecommendations or GetPersonalizedRanking API, or while generating recommendations in batch mode through a batch inference job.

This post walks you through setting up and using item and user metadata-based recommendation filters in Amazon Personalize.

Prerequisites

To define and apply filters, you first need to set up the following Amazon Personalize resources. For instructions on the Amazon Personalize console, see Getting Started (Console).

Create a dataset group.

Create an Interactions dataset using the following schema and import data using the interactions-100k.csv data file:

{
	"type": "record",
	"name": "Interactions",
	"namespace": "com.amazonaws.personalize.schema",
	"fields": [
		{
			"name": "USER_ID",
			"type": "string"
		},
		{
			"name": "ITEM_ID",
			"type": "string"
		},
		{
			"name": "EVENT_VALUE",
			"type": [
				"null",
				"float"
			]
		},
		{
			"name": "TIMESTAMP",
			"type": "long"
		},
		{
			"name": "EVENT_TYPE",
			"type": "string"
		}
	],
	"version": "1.0"
}

Create an Items dataset using the following schema and import data using the csv data file:

{
    "type": "record",
    "name": "Items",
    "namespace": "com.amazonaws.personalize.schema",
    "fields": [
        {
            "name": "ITEM_ID",
            "type": "string"
        },
        {
            "name": "GENRES",
            "type": "string",
            "categorical": true
        },
        {
            "name": "CREATION_TIMESTAMP",
            "type": "long"
        }
    ],
    "version": "1.0"
}

Create a solution using any recipe. In this post, we use the aws-hrnn recipe.
Create a campaign.

Creating your filter

Now that you have set up your Amazon Personalize resources, you can define and test custom filters.

Filter expression language

Amazon Personalize uses its own DSL called filter expressions to determine which items to exclude or include in a set of recommendations. Filter expressions are scoped to a dataset group. You can only use them to filter results for solution versions (an Amazon Personalize model trained using your datasets in the dataset group) or campaigns (a deployed solution version for real-time recommendations). Amazon Personalize can filter items based on user-item interaction, item metadata, or user metadata datasets.

The following are some examples of filter expressions by item:

To remove all items in the "Comedy" genre, use the following filter expression:
```
EXCLUDE ItemId WHERE item.genre in ("Comedy")
```
To include items with a "number of downloads" less than 20, use the following filter expression:
```
INCLUDE ItemId WHERE item.number_of_downloads < 20
```

The following are some examples of filter expressions by interaction:

To remove items that have been clicked or streamed by a user, use the following filter expression:
```
EXCLUDE ItemId WHERE interaction.event_type in ("click", "stream")
```
To include items that a user has interacted with in any way, use the following filter expression:
```
INCLUDE ItemId WHERE interactions.event_type in ("\*\")
```

You can also filter by user:

To exclude items where the number of downloads is less than 20 if the current user’s age is over 18 but less than 30, use the following filter expression:
```
EXCLUDE ItemId WHERE item.number_of_downloads < 20 IF CurrentUser.age > 18 AND CurrentUser.age < 30
```

You can also chain multiple expressions together, allowing you to pass the result of one expression to another in the same filter using a pipe ( | ) to separate them:

The following filter expression example includes two expressions. The first expression includes items in the "Comedy" genre, and the result of this filter is passed to another expression that excludes items with the description "classic":
```
INCLUDE Item.ID WHERE item.genre IN (“Comedy”) | EXCLUDE ItemID WHERE item.description IN ("classic”)
```

For more information, see Datasets and Schemas. For more information about filter definition DSL, see Filtering Recommendations.

Creating a filter on the console

You can use the preceding DSL to create a custom filter on the Amazon Personalize console. To create a filter, complete the following steps:

On the Amazon Personalize console, choose Filters.
Choose Create filter.
For Filter name, enter the name for your filter.
For Expression, select Build expression.

Alternatively, you can add your expression manually.

To chain additional expressions with your filter, choose +.
To add additional filter expressions, choose Add expression.
Choose Finish.

Creating a filter takes you to a page containing detailed information about your filter. You can view more information about your filter, including the filter ARN and the corresponding filter expression you created. You can also delete filters on this page or create more filters from the summary page.

You can also create filters via the createFilter API in Amazon Personalize. For more information, see Filtering Recommendations.

Applying your filter to real-time recommendations on the console

The Amazon Personalize console allows you to spot-check real-time recommendations on the Campaigns page. From this page, you can test your filters while retrieving recommendations for a specific user on demand. To do so, navigate to the Campaigns tab; this should be in the same dataset group that you used to create the filter. You can then test the impact of applying the filter on the recommendations.

Recommendations without a filter

The following screenshot shows recommendations returned with no filter applied.

Recommendations with a filter

The following screenshot shows results after you remove the "Action" genre from the recommendations by applying the filter we previously defined.

If we investigate the Items dataset provided in this example, item 546 is in the genre "Action" and we excluded the "Action" genre in our filter.

This information tells us that Item 546 should be excluded from recommendations. The results show that the filter removed items in the genre "Action" from the recommendation.

Applying your filter to batch recommendations on the console

To apply a filter to batch recommendations on the console, follow the same process as real-time recommendations. On the Create batch inference job page, choose the filter name to apply a previously created filter to your batch recommendations.

Applying your filter to real-time recommendations through the SDK

You can also apply filters to recommendations that are served through your SDK or APIs by supplying the filterArn as an additional and optional parameter to your GetRecommendations calls. Use "filterArn" as the parameter key and supply the filterArn as a string for the value. filterArn is a unique identifying key that the CreateFilter API call returns. You can also find a filter’s ARN on the filter’s detailed information page.

The following example code is a request body for the GetRecommendations API that applies a filter to a recommendation:

{
    "campaignArn": "arn:aws:personalize:us-west-2:000000000000:campaign/test-campaign",
    "userId": "1",
    "itemId": "1",
    "numResults": 5,
    "filterArn": "arn:aws:personalize:us-west-2:000000000000:filter/test-filter"
}

Applying your filter to batch recommendations through the SDK

To apply filters to batch recommendations when using a SDK, you provide the filterArn in the request body as an optional parameter. Use "filterArn" as the key and the filterArn as the value.

Summary

Customizable recommendation filters in Amazon Personalize allow you to fine-tune recommendations to provide more tailored experiences that improve customer engagement and conversion according to your business needs without having to implement post-processing logic on your own. For more information about optimizing your user experience with Amazon Personalize, see What Is Amazon Personalize?

About the Author

Matt Chwastek is a Senior Product Manager for Amazon Personalize. He focuses on delivering products that make it easier to build and use machine learning solutions. In his spare time, he enjoys reading and photography.

AWS Machine Learning Blog