Fueling real-time recommendations using a graph-based approach for complex data relationships

In today's modern era, we are surrounded by endless options—whether it's the brand of soda we drink, the sunglasses we wear, or the movies we watch. Yet despite remarkable technological advancements, many popular websites still offer one-size-fits-all recommendations and fail to capture our unique preferences, leaving us to search for needles in haystacks of choices. By harnessing data from multiple sources, it's now possible to provide users with a truly personalized experience. In this article, you'll learn how to leverage the power of data to provide your users with recommendations they will genuinely value.

Let’s consider the following situation for our example solution. Imagine you are a software architect who recently transferred to the sought-after Idea Incubation department of a highly successful XXXL enterprise organization. Your legendary company does practically every kind of e-commerce business, from a real estate website to a jewelry web store to a luxury retail shopping website to a social media website. You are tasked with building the next version of the recommendation engine for the e-commerce website that sells luxury designer labels.

You are tasked with building a system that provides personalized customer recommendations to high net worth (HNW) individuals. The current implementation just does not work; 85% of its recommendations are ignored by customers. The product team found that recommendations are often out of customers’ comfort range and do not consider their brand affinity.

You have been tasked to build a 360-degree view of the target customer, including taking data from multiple sources so you can recommend products to customers within the customer’s range while considering brand affinity.

Let’s figure out how to recommend the right luxury items to the target customer persona.

In the past, you’ve probably built recommendation systems for a single website. For building recommendation systems, you likely used the standard collaborative filtering approach. At a high level, collaborative filtering uses item, user, and item-user interaction data. In this approach, you probably built a database of user preferences, grouped users based on similar preferences, and then calculated similarities between users to make recommendations based on similar users' preferences.

If this approach works for everyone else, why doesn’t it work for your new target (HNW) customer base? The HNW customer exhibits a different type of buying pattern. They make high-value, low-frequency purchases, and their purchases are extremely personalized. Think about these examples:

1. Purchasing a $20,000 art piece. This type of purchase isn’t based on an ad click but more likely proximity of the gallery, familiarity with the artist, and social status.
2. Purchasing a $500 bag. When the shopper purchases new Hermes bags every now and then, it’s likely this particular purchase was a gift, perhaps for the nanny, and shouldn’t be considered part of a repeatable brand affinity.
3. Purchasing a 3-carat diamond ring a week before Valentine's Day. Such a jewelry purchase near a romantic holiday might not mean the shopper will buy one every year at that time, especially if it’s an engagement ring.

Clearly, we need some elite platinum filtering system for our HNW customers; the standard collaborative filtering approach won’t work. What will keep the customer happy and purchasing? Here are some alternate popular approaches widely used for building recommendation systems:

1. Content-based filtering
   Content-based filtering takes a two-pronged approach: First, you build an item profile by associating relevant features with items; second, you build a user profile based on items the user has interacted with. Finally, you match and recommend items based on the characteristics of items the user has interacted with.
   
   Although the solution gives highly personalized recommendations, the approach does not work for HNW customers, as it suffers from overspecialization.
   
   
2. Deep learning models
   Another popular option is to train a deep learning model. ML models handle high dimensional data well, understand contextual signals, and can adapt to changing user patterns.
   
   Deep learning models require massive training data. Since HNW transactions are limited, finding such data is challenging.
   

It’s clear that with this type of customer, you can’t take a simplistic, boilerplate approach. You need to understand the customer’s mindset, brand affinities, and shopping expectations. You need to think about the customer from multiple angles:

1. What are the customer’s preferences?
2. Does the customer care about the brand? Do they stick to the same brand while repeatedly purchasing other items in the same category?
3. Are there purchases or items that the customer prefers to purchase at a brick-and-mortar store?

Since your organization practically knows it all, you have the ability to really give your HNW customers the personalized experience they expect. You must consider their lifestyle, preferences across different platforms, brand affinity, and real-time browsing behavior. You must aggregate data relevant to the task from multiple sources.

Here is the data you could aggregate data for your HNW customers:

1. Luxury item interaction data from the website, including item characteristics and interactions data.
2. Social media interaction data, including posts, likes, and comments, if they tag or feature a luxury brand.
3. User characteristic data.

Being an architect in the incubator group gives you easy access to de-identified data from different departments that you can join with each other. Now, you must determine where to gather and store the subset containing target customer data. You could store it in a data warehouse, but wait, is that really the right way? How many tables are you going to join? How complex would your queries be, and how long would they run? Could it serve as a place from which you can return your recommendations in a matter of milliseconds? Is it really the right solution? What if you want to add new data to your analysis? What if the data is not structured?

A relational database won’t work for you because of the rigidity of the schema. You not only need an adaptable, schema-less storage location that is optimized for traversals, but you also need a processing engine with a distributed architecture that can scale based on demand. Enter the graph database! The graph database was built to manage large and growing datasets with complex interconnections. They support flexible queries and are the go-to solution to store user data for modeling complex relationships.

Here is an example of what a graph looks like. The graph depicts that a person, an actor, bought a ring from Tiffany & Co.

The graph consists of three nodes – a Person node, a Ring node, and a Jeweler. The leftmost node in the graph has three properties – Name (James Smith), Type (HNW), and Occupation (Actor). The left node is linked with a node labeled as Ring via the BOUGHT relationship. The relationship has a property called date with a value as ‘12/02/2024. You can use Cypher query language to extract information from the graph.

Here is a sample query that extracts the person and their corresponding purchase:

MATCH (p:Person)-[:BOUGHT]->(t:Ring)
RETURN p,t

Graphs are the perfect way to store and process data for the HNW recommendations use case. A graph database enables you to gather, store, and query luxury item interaction data, social media data, and user characteristic data. However, as an enterprise organization, if you don’t want to hire a team, you need a fully managed solution that meets your SLA requirements and with a high availability guarantee. That is, assuming you don’t want service interruptions for your recommendation engine!

If you are like me, you pay close attention to industry trends while exploring new products. You come across the Forrester Wave report for databases and see Neo4j recognized as a Strong Performer in the Forrester Wave™ Q3 2024 and in the Gartner Magic Quadrant for 2024 as a Visionary. As you explore Neo4j AuraDB, a fully managed graph database, features fit this use-case such as a 99.95% uptime SLA, RBAC, AWS PrivateLink support, ACID compliance, and a built-in data visualizer that helps you find hidden connections and relationships using Bloom and Neo4j Browser. You especially like features such as index-free adjacency. Index-free adjacency means a node references neighbor nodes, which makes accessing relationships and related data a matter of a memory pointer lookup. This really speeds up queries, as relationships are the star of graph queries.

Based on your research, you decide to try Neo4j AuraDB in AWS Marketplace.

After you build a data model based on queries you want to execute, you pre-process the data before loading it into the graph database. First, you process the social media data with Amazon Comprehend; if the post the customer liked, wrote, or left a positive comment on mentions a brand name or tags a brand, you perform sentiment analysis, then associate it with the item, and add an ENDORSED relationship while loading it into the database. If a picture the customer posted has a brand, you perform object detection using Amazon Rekognition Custom Labels, extract the brand name, and add an ENDORSED relationship.

Now you’ve got some data on luxury items your customers like. Your next step is to transform the luxury data and load it into the Neo4j database. You convert people and items into nodes and add properties to these nodes. You create specific properties if a person's attributes match the item's attributes. For example, if a person is a man who bought a men’s watch, you can create a PURCHASED_FOR_SELF relationship.

Once you have identified how to process and load the data into the database, it’s time to analyze the data you’ve collected. Here is the sample data from the fashion accessories category for a de-identified individual.

At a glance, you can see that this individual has a taste for expensive designer fashion accessories, but why not let the database do the work here? Let’s find out more about this individual with Cypher queries.

1. What brands has the customer purchased within the fashion accessories category?

The result of this query shows us some nuance in the HNW person’s shopping habits: the customer prefers Louis Vuitton for accessories and Hermes for bags. So, it’s a good bet to recommend recently released Hermes bags that the customer does not own.

2. What brands does the customer endorse on social media?

This customer writes about Louis Vuitton and Cartier on social media, so you can assume a brand affinity for both.

3. A joint query – Returns brands customer has an affinity with:

If you join the two queries, you will get a single list of products the customer has an affinity with.

That query returns Hermes, Cartier, Tiffany & Company, and Louis Vuitton.

4. Return recently added products from brands the customer has an affinity with

Now that you have a list of preferred brands for the customer let’s ask for recently released products from the customer’s preferred brand list that the customer does not already own.

Now that’s a powerful result! If your recommendation engine were to recommend these items to this HNW shopper, you’d likely get a sale or two!

You still have a long way to go. Even wealthy folks have spending limits and seasonal spending patterns. Here are some improvements you can make to this query to make it better:

1. Add a check around the item's price. Ensure it is within two deltas of the average price of the products the customer bought in the subcategory.
2. Identify influencers the customer engages with. Once you note the key influencers they interact with, do object detection to identify brands and products they endorse and run sentiment analysis on posts to identify sentiment about brands or specific products.
3. Identify seasonal patterns. Does this shopper typically spend on gifts around Valentine’s Day? Is there an annual peak that could be a partner or sibling’s birthday or an anniversary? What is their spending like around Christmas or Mother’s Day?
4. Bulk purchases. Is there a pattern of bulk purchases around summer or Christmas?
5. Identify trending social media. Is this shopper influenced by trends on platforms like Instagram or TikTok? For instance, are they purchasing items popularized by viral challenges or influencers during specific periods?

By analyzing sample data and extracting patterns, you can build a query that returns products customers would be most comfortable with. You can use this information to improve the quality of recommendations made to the customer.

Congratulations! You just learned how to use a graph database to extract complex relationships so you can build a successful, targeted recommendation engine for high-net-worth individuals seeking luxury items. Now consider how this technology combination can help you solve for your own use-cases.

Here are some key takeaways to help you get started:

1. Understand your customer and see what data is available (or can be procured from a third party) necessary for making better recommendations.
2. Understand the requirements and build a clear data model that fits your query requirements.
3. Use Neo4j AuraDB to build a proof of concept (POC) that implements a better recommendation engine.
4. Use best practices such as creating indexes and tuning queries to improve the performance of your queries.
5. Innovate on behalf of your customers.

You can get started by trying Neo4j AuraDB in AWS Marketplace with pay-as-you-go billing through your AWS account.