Capgemini’s Edge-Capable Targeted Campaigns for Popup Stores Using Deep Learning

By Diptangshu Chakrabarty, Sr. Manager, Architect – Capgemini
By Hemant Sharma and Prateek Agrawal, Partner Solutions Architects – AWS
By Pushpal Mukherjee, Manager – Capgemini
By Rohit Barde, Consultant – Capgemini

Capgemini

As direct to customer (D2C) gains popularity among retailers, there’s an increasing need to mix online and offline experiences to improve customer engagements and sentiment. One such popular channel is popup stores.

Popup stores are temporary physical outlets set up in key locations like subway stations, parks, and events, to carry out customer engagement activities. This includes increasing brand awareness, introducing new products, and conducting user surveys.

In this post, we will share an architectural reference for a Capgemini solution that uses Amazon Web Services (AWS) to help retailers engage with customers in a smart way. The solution leverages deep learning to enhance the customer experience through gamification and provides key insights and marketing leads to retailers.

Capgemini is an AWS Premier Tier Services Partner and Managed Cloud Services Provider (MSP) that is at the forefront of innovation to address a breadth of client opportunities across cloud, digital, and platforms.

Driving Business Growth Using Deep Learning in Popup Stores

Based on a survey conducted by Business Insider, 80% of global retail companies that have opened a popup store said it was successful. More so, 58% are likely to use the tactic again.

Global retailers use the concept to realize several benefits, including improved market visibility and understanding customer needs.

Figure 1 – Global retailers benefit from popup stores.

Equipped with deep learning capabilities, Capgemini’s “in-a-box” hardware and software integrated solution can be set up to drive many of these benefits. Leveraging AWS DeepLens or AWS Panorama, it enables retailers to perform contextual marketing using personalization and gamification. The solution provides a report dashboard that shows key metrics to the business user about the popup store, including:

• Lead capture reports: How many visitors were interested in the campaign and redeemed the awarded coupons, and how many backed off.
• Foot fall heatmap: This report helps retailers determine the dominating demographic of visitors where the popup store is installed.
• User density: This report plots the visit density of visitors in the popup store over different product sections.

These reports help retailers make decisions such as:

• Effectiveness of product campaigns by gathering foot-traffic conversion information in retail outlets.
• Best location for installing the popup store so the maximum number of visitors can be reached.
• Campaign performance with respect to targeted demographics.
• User journeys that help drive value in the product and reward users for their interest.

User Journey

The popup store user journey can be split into three phases:

1. Interest: One or many customers visit the popup store.
2. Interaction: Deep learning-enabled edge camera captures the image and recognizes the visitor’s demographics, and a targeted product campaign with a specific QR code is displayed over a HDMI display in the popup store.
3. Incentive: The customer can scan the QR code from the display using their mobile device and redeem the coupon from the online website. As a result, a lead is generated and captured in a customer relationship management (CRM) system such as Salesforce.

Figure 2 – User journey.

Architecture

This popup store is a product campaign-based solution that has been implemented as a combination of hardware and software with deep learning capabilities. The physical components of the store are shown in Figure 3.

Figure 3 – Logical flow diagram.

The following physical components need to be installed in the store/kiosk:

• AWS DeepLens or AWS Panorama: This captures the video feed of store visitors. It runs an edge AI detection algorithm to identify the demographics and sends it to the backend system to fetch relevant campaign information.
• Large HDMI display: This is used to run personalized campaigns in the store. These can be in form of slide shows or product videos. At the end of the video, a QR code may be displayed for visitors to scan as an incentive. This display can be used in more engaging gamification-oriented campaigns.
• Progressive Web App (PWA): This PWA is connected to the HDMI display and shows the relevant campaign marketing content in real-time and based on the demographic details captured by the deep learning model.

The architectural diagram in Figure 4 represents the process view of data flow from an organization’s CRM system, and the various AWS services used to implement the popup solution.

Figure 4 – Architecture diagram.

Let’s look at the solution architecture:

1. A model is trained to predict demographic information in Amazon SageMaker, which is stored in an Amazon Simple Storage Service (Amazon S3) bucket and imported into AWS DeepLens or AWS Panorama. SageMaker helps data scientists and developers prepare, build, train, and deploy high-quality machine learning (ML) models quickly by bringing together a broad set of capabilities purpose-built for ML.
2. While the user walks into the popup store, an AWS DeepLens camera or AWS Panorama installed in the store captures the video feed and is processed through the model. AWS DeepLens is a fully programmable video camera that comes with tutorials, code, and pre-trained models designed to expand deep learning skills.
3. The inference results by the AWS DeepLens or AWS Panorama model are then written to a csv file in an S3 bucket.
4. An AWS Lambda function is triggered when a new entry is made in Amazon S3. This function aggregates the inference results and acts as a data resolver for AWS AppSync, a fully managed service that makes it easy to develop GraphQL APIs by handling the heavy lifting of securely connecting to data sources like Amazon DynamoDB and Lambda.
5. PWA is subscribed to the data from the AppSync API which gets updated with the new data received through the AWS DeepLens inference results. The product catalog displayed on the signage media player changes according to customer demographics. Media player also shows campaign-specific coupons QR codes targeted to the respective audience. Users can redeem the coupons by providing their email address.
6. Leads are captured and stored in S3, which is then synced with Salesforce using Amazon AppFlow, a fully managed integration service that enables you to securely transfer data between software-as-a-service (SaaS) applications and AWS services in just a few clicks. Amazon AppFlow also sync up campaign codes stored in s3 bucket from the Salesforce CRM.
7. The raw inference results are stored in Amazon S3 and can be further transformed and stored back in another S3 bucket. Amazon Athena queries the transformed data stored in S3 buckets, and all the analysis, reports, or dashboards will be shown in Amazon QuickSight. Athena is an interactive query service that makes it easy to analyze data in S3 using standard SQL, while QuickSight is a cloud-scale business intelligence (BI) service that delivers easy-to-understand insights.
8. Lead capture data and AWS DeepLens or AWS Panorama inference results are visualized using different charts and dashboards in QuickSight. The dashboards contain the latest frames captured by the camera.

Building the Solution

The AWS DeepLens face detection model has been used and customized accordingly to detect faces and store them in Amazon S3 with the person’s demographic information. It can detect a single face as well as multiple faces that are appearing on camera.

The customized model function is incorporated as a AWS DeepLens project. When a user appears in front of the AWS DeepLens camera, it continuously captures the frames, generates the demographic details using the deep learning model, and stores the person’s demographic details in the allocated Amazon S3 bucket.

Application Screenshots

Figure 5 shows the default screen without any marketing campaign.

Figure 5 – Default display. 

The screen in Figure 6 displays a targeted marketing campaign to a user whose demographics has been captured using AWS DeepLens or AWS Panorama. The user will be able to scan the QR code and will be redirected to the next screen.

Figure 6 – Display targeted campaign.

The user scans the QR code to access the offer, and then redeems the coupon by entering their email address.

Figure 7 – Redeem coupon by scanning QR code.

Figure 8 shows the engagement analytics report in the Amazon QuickSight dashboard.

Figure 8 – Popup store engagement report for retailers.

Conclusion

The popup store campaign-based solution can be applied to multiple industries, including fashion, cosmetic products, food and beverage, and more. Realizing its potential, the Capgemini innovation team created a ready-made solution that retailers can use to set up a popup store quickly.

It can be integrated with the retailer’s IT landscape very easily, and the Capgemini’s popup store solution in this post uses AWS DeepLens as an edge camera device. However, the same architecture can be used for AWS Panorama devices as well.

Leveraging deep learning-enabled camera devices in the context of popup stores enables retailers to come up with interesting engagement stories. The dashboards can help them understand customers better and follow a test-and-learn approach for product promotion.

To learn more, contact the Capgemini team.

.
.

Capgemini – AWS Partner Spotlight

Capgemini is an AWS Premier Tier Services Partner and Managed Cloud Services Provider (MSP) that is at the forefront of innovation to address a breadth of client opportunities across cloud, digital, and platforms.

Contact Capgemini | Partner Overview