Generate suggestions for leisure activities in real time with Amazon Neptune

DoGet App is a mobile application that connects friends for sharing in-person moments together.

Suggestions for activities to engage in with friends are presented to users in card deck format: swiping up indicates no interest in an activity, and swiping down indicates interest and prompts a follow-up on when a user is available (such as tomorrow or this week). DoGet App then matches friends interested in sharing the same experience at the same time.

In this post, we explore how DoGet App adopted a graph data model backed by Amazon Neptune that served their use case and enabled real time activity suggestions and matchmaking. Next to this purpose-built database approach, their serverless architecture uses AWS AppSync, AWS Amplify, AWS Lambda, and Amazon Simple Storage Service (Amazon S3) and allowed them to scale during launch and onboard thousands of new users per day. By building on top of AWS managed services, DoGet App could significantly reduce time to market with only 6 months between start of development and launch in Q4 2022, while at the same time consolidating development effort prior to and operational overhead after launch.

Challenges of designing and launching the application

DoGet App faced several challenges when designing their application.

Firstly, many matchmaking applications pre-generate recommendations for faster load times without taking into account real time states. DoGet App, on the other hand, goes beyond past habits and preferences when generating suggestions. Content elements are generated live in order to incorporate a user’s current location, weather conditions, and responses from friends.

To tackle this challenge, DoGet App uses Amazon Neptune with a graph data model tailored to their algorithms. A consistent live representation of the overall state of all the user’s relationships, interests, and interactions is maintained in the Neptune graph database, based on which suggestions and matches are created in real time.

In addition to choosing the right data model for their use case, DoGet App needed to onboard their complete user base from scratch and therefore required a scalable solution to accommodate for unpredictable load and increases in traffic. They also needed an operational model that allowed them to iterate fast and build out the graph data model and corresponding algorithms based on new insights gained while growing their user base.

Overview of solution

In this section, we present a high-level overview of the application architecture before diving deeper into the graph data model and algorithms in the subsequent sections.

The following figure shows an architecture diagram, including the application front-end and back-end as well as data analytics capabilities.

The mobile application front-end uses AWS Amplify client libraries to connect to back-end resources in the AWS Cloud. A typical request flow is as follows:

1. Mobile clients fetch static front-end assets from Amazon S3.
2. After authenticating to Amazon Cognito,
3. they gain access to the back-end APIs provided by AWS AppSync.
4. Requests are subsequently processed by AWS Lambda functions
5. that query information from Amazon DynamoDB and Amazon Neptune and generate responses to API requests.
6. In the process of serving these requests, analytics data is stored to a data lake in Amazon S3,
7. from where it can be analyzed with Amazon Athena.

Graph data model

At the core of DoGet App’s functionality is a purpose-built graph database backed by Amazon Neptune, a fully managed graph database service. With Neptune, DoGet App could focus on the actual data model and algorithm design relevant to their application while offloading operational tasks such as maintenance, backups, and scaling.

Let’s dive deeper into the graph data model to understand how it benefits DoGet App’s use case. Fundamentally, nodes in the graph represent entities in the application, whereas edges describe the relationship between them. The following figure illustrates common entities in the app’s data model.

This example shows three person nodes representing the users Lisa, Sophie, and Benny. Edges between person nodes represent their friendship relations. Then there is one activity node (Explore unknown paths in the surroundings by bike), which is related to three topic cluster nodes (Outdoor, Nature Lover, and Hiking).

The app’s data model also incorporates location information in the form of geo hashes (no exact location information is required for that approach). This information is represented in the graph data model by region nodes, as shown in the following figures.

Locations are related to suitable activities for that area and in terms of weather, as well as to users in terms of their current and past location. Edge weights are used to model the strength of a relation. Given that DoGet App aims to connect users in-person, location information is particularly relevant when generating suggestions and matchmaking, especially because it can also be used to incorporate weather information.

Generating suggestions in real time

Whenever a user opens DoGet App for the first time of the day, a set of algorithms generates a card deck of 30–50 activity suggestions. All these algorithms either directly or indirectly use the real time graph representation stored in Neptune. A mix of different algorithms is used for a balanced set of suggestions. For example, some algorithms aim at inspiring users with new activities, some favor activities recently liked by friends to increase the likelihood of a match, while other algorithms use the strongest relations of a user to other entities in the graph.

Let’s explore an example of an algorithm that focuses on the latter. Starting from a user, this algorithm traverses various paths to activities that a user is connected to in the graph (both directly and indirectly). Depending on the strength of a connection and on its type (whether it is direct or indirect, for example via a friend or a topic cluster), the algorithm computes metrics for these activities and generates suggestions based on that, as illustrated by the following figure.

Although this algorithm incorporates all types of connections, others emphasize particular aspects, for example, a user’s current location for providing suggestions of regional activities and current events in a region.

Irrespective of the algorithm, the graph data structure is ideal for DoGet App to generate suggestions in real time. Moreover, the graph is well suited for matchmaking: whenever a user indicates interest in an activity, potential matches are close by in the graph and include all activities already liked by their friends, which can be queried efficiently.

Incorporating new content and gaining deeper insights

Most relations and corresponding edge weights in the graph are established over time according to user preferences and past behavior, whereas others are explicitly modeled. For example, when a user indicates interest by liking an activity, their relation to it and related nodes is strengthened, thereby increasing the weight of the connecting edge. Especially during launch, generating suggestions for new content elements proved challenging because they are initially not well connected to the rest of the graph. As described in the previous section, one approach to integrate new activities is via a mix of algorithms.

Another approach that DoGet App is currently evaluating is using Amazon Neptune ML, which allows them to go beyond relations in the graph and also base suggestions on, for example, similarities in the text description of activities.

Finally, DoGet App is constantly trying to improve their content offering in terms of activity suggestions. Towards that end, app usage data is stored in Amazon S3 and can be analyzed with Athena to gain insights. However, because the edge weights in the graph are dynamically adapted to reflect preferences and past usage, DoGet App can also run analytics queries based on their graph by examining the strongest edges. For example, to find the three most popular activities for a particular region, we can issue a graph query and order by edge strength of connected activities:

%% gremlin 
g.V('target-region')  
 .outE().has(label,'goodFor') 
 .order().by('strength',desc).limit(3) 
 .inV().unfold().values('name')

In conjunction with traditional analytics, DoGet App uses these insights to improve their activity portfolio.

Conclusion

Amazon Neptune enabled DoGet App to build their application around a graph data model without the complexities involved with managing their own graph database management system. The data model and related algorithms helped them create an outstanding user experience: suggestions based on the graph algorithms yield an acceptance rate of about 25 percent. Moreover, their matchmaking algorithms result in daily matches for users with active friends. By using AWS managed services, DoGet App can generate business value fast by focusing on their differentiating skills such as application design and content creation.

Did this post inspire you to evaluate a graph data model for your use case? Then check out the Amazon Neptune product page and get hands-on with Neptune samples on GitHub.

About the authors

Michael Meidlinger is a Sr. Solutions Architect at AWS where he works with Enterprise customer in regulated industries. He holds a PhD in telecommunications engineering with a background in signal processing, statistics, networking, embedded systems development and Linux system administration. Before joining AWS, he worked on mission critical compute systems and safe cloud computing concepts.

Nils Müller was born in Austria and studied business engineering in Wiener Neustadt. During his studies in 2000, he founded the company 1000PS Internet GmbH. With the brand 1000PS, the company became the world’s leading content provider for motorcycle topics. In response to the COVID-19 pandemic, he came up with the idea for the DoGet App.