Discovering Hot Topics Using Machine Learning

What does this AWS Solution do?

The Discovering Hot Topics Using Machine Learning solution identifies the most dominant topics associated with your products, policies, events, and brands. This helps you to react quickly to new growth opportunities, address negative brand associations, and deliver a higher level of customer satisfaction for your business. In addition to helping you understand what your customers are saying about your brand, this solution gives you insights into topics that are relevant to your business.

This solution deploys an AWS CloudFormation template to automate data ingestion from these sources:

  • Twitter
  • RSS news feeds
  • YouTube comments tied to videos
  • Reddit (comments from subreddits of interest)
  • Custom data in JSON or XLSX format

Using the Reddit API, the solution ingests comments from subreddits of interest. The input parameters to the solution includes a list of subreddits to follow for new comments. The comments then undergo NLP analysis to build the Amazon QuickSight dashboard.

Expand the image to learn about how the solution answers the question: "What are the most dominant topics related to my product or service?"


Secure one-click deployment

Provide a secure one-click deployment using an AWS CloudFormation template developed with the AWS Well-Architected Framework methodologies.

Near real-time analytics

Ingest streaming data containing text and images, then analyze them in near real-time. Perform topic modeling to detect dominant topics and identify the terms that collectively form a topic from within customer feedback.

Multi-lingual data ingestion

Use Amazon Translate to ingest data in multiple languages. Identify the sentiment of what customers are saying and use contextual semantic search to understand the nature of online discussions.

Pre-built QuickSight dashboard

Launch the pre-built Amazon QuickSight dashboard to visualize the solution's large-scale customer analyses. Identify insights in near real-time to better understand context, threats, and opportunities almost instantly.

AWS Solution overview

The diagram below presents the serverless architecture you can automatically deploy using the AWS Solution's implementation guide and accompanying AWS CloudFormation template.

Discovering Hot Topics Using Machine Learning architecture

The AWS CloudFormation template automatically deploys AWS Lambda functions, Amazon Simple Storage Service (Amazon S3) buckets, Amazon Kinesis Data Streams, Amazon Simple Queue Service (Amazon SQS) dead-letter-queue (DLQ), Amazon Kinesis Data Firehose, AWS Step Functions workflows, AWS Glue tables, and Amazon QuickSight resources in your account.

The architecture of the solution includes the following key components and workflows:

1. Ingestion – Social media and RSS feed ingestion and management using Lambda functions, Amazon DynamoDB, and Amazon EventBridge. For detailed reference architecture diagrams for Twitter, YouTube comments, RSS news feeds, and custom ingestion using an Amazon S3 bucket, refer to the implementation guide.

2. Data stream – The data is buffered through Amazon Kinesis Data Streams to provide resiliency and throttle incoming requests. The Data Streams have a configured DLQ to catch any errors in processing feeds.

3. Workflow – Consumer (Lambda function) of the Kinesis Data Streams initiates a Step Functions workflow that orchestrates Amazon Machine Learning capabilities including: Amazon Translate, Amazon Comprehend, and Amazon Rekognition.

4. Integration – The inference data integrates with the storage components through an event-driven architecture using Amazon EventBridge. EventBridge allows further customization to add additional targets by configuring rules.

5. Storage and visualization – A combination of Kinesis Data Firehose, Amazon S3 buckets, AWS Glue tables, Amazon Athena, and Amazon QuickSight.

These components are built using the AWS Well-Architected Framework, and the AWS Well-Architected Pillars of Operational excellence, Security, Reliability, Performance efficiency, and Cost optimization—ensuring secure, high-performing, resilient, and efficient infrastructure.

Discovering Hot Topics Using Machine Learning

Version: 2.1.0
Last updated: 12/2022
Author: AWS

Estimated deployment time: 10 min

Use the button below to subscribe to solution updates.

Note: To subscribe to RSS updates, you must have an RSS plug-in enabled for the browser you are using.  

Did this Solutions Implementation help you?
Provide feedback 
Solving with AWS Solutions: Discovering Hot Topics Using Machine Learning
Understanding 'Voice of Customer' with AWS Solutions
AWS Architecture Blog
Discovering Hot Topics Using Machine Learning

This blog post teaches readers how to use the Discovering Hot Topics Using Machine Learning solution to draw insights from social media feeds in order to take advantage of rapidly emerging growth opportunities, to address negative sentiment, and to improve customer satisfaction. As an illustration, we walk through a business use case in the Media & Entertainment industry.

Read the full blog post 
Build icon
Deploy an AWS Solution yourself

Browse our library of AWS Solutions to get answers to common architectural problems.

Learn more 
Find an APN partner
Find an AWS Partner Solution

Find AWS Partners to help you get started.

Explore icon
Explore Guidance

Find prescriptive architectural diagrams, sample code, and technical content for common use cases.

Learn more