Overview
The Discovering Hot Topics Using Machine Learning solution identifies the most dominant topics associated with your products, policies, events, and brands. This helps you to react quickly to new growth opportunities, address negative brand associations, and deliver a higher level of customer satisfaction for your business. In addition to helping you understand what your customers are saying about your brand, this solution gives you insights into topics that are relevant to your business.
This solution deploys an AWS CloudFormation template to automate data ingestion from these sources:
- RSS news feeds
- YouTube comments tied to videos
- Reddit (comments from subreddits of interest)
- Custom data in JSON or XLSX format
Benefits
Provide a secure one-click deployment using an AWS CloudFormation template developed with the AWS Well-Architected Framework methodologies.
Ingest streaming data containing text and images, then analyze them in near real-time. Perform topic modeling to detect dominant topics and identify the terms that collectively form a topic from within customer feedback.
Use Amazon Translate to ingest data in multiple languages. Identify the sentiment of what customers are saying and use contextual semantic search to understand the nature of online discussions.
Launch the pre-built Amazon QuickSight dashboard to visualize the solution's large-scale customer analyses. Identify insights in near real-time to better understand context, threats, and opportunities almost instantly.
Technical details
You can automatically deploy this architecture using the implementation guide and the accompanying AWS CloudFormation template.
These components are built using the AWS Well-Architected Framework, and the AWS Well-Architected Pillars of Operational excellence, Security, Reliability, Performance efficiency, and Cost optimization—ensuring secure, high-performing, resilient, and efficient infrastructure.
Step 1 - Ingestion
AWS Lambda functions, Amazon DynamoDB, and Amazon EventBridge provide social media and RSS feed ingestion and management. For detailed reference architecture diagrams for YouTube comments, RSS news feeds, and custom ingestion using an Amazon Simple Storage Service (Amazon S3) bucket, refer to the implementation guide.
Step 2 - Data stream
The data is buffered through Amazon Kinesis Data Streams to provide resiliency and throttle incoming requests. The Kinesis Data Streams have a configured DLQ to catch any errors in processing feeds.
Step 3 - Workflow
Consumer (Lambda function) of the Kinesis Data Streams initiates an AWS Step Functions workflow that orchestrates Amazon machine learning capabilities including: Amazon Translate, Amazon Comprehend, and Amazon Rekognition.
Step 4 - Integration
The inference data integrates with the storage components through an event-driven architecture using EventBridge. EventBridge allows further customization to add additional targets by configuring rules.
Step 5 - Storage and visualization
A combination of Amazon Kinesis Data Firehose, S3 buckets, AWS Glue tables, Amazon Athena, and Amazon QuickSight provide storage and visualization.
Step 1 - Ingestion
AWS Lambda functions, Amazon DynamoDB, and Amazon EventBridge provide social media and RSS feed ingestion and management. For detailed reference architecture diagrams for YouTube comments, RSS news feeds, and custom ingestion using an Amazon Simple Storage Service (Amazon S3) bucket, refer to the implementation guide.
Step 2 - Data stream
The data is buffered through Amazon Kinesis Data Streams to provide resiliency and throttle incoming requests. The Kinesis Data Streams have a configured DLQ to catch any errors in processing feeds.
Step 3 - Workflow
Consumer (Lambda function) of the Kinesis Data Streams initiates an AWS Step Functions workflow that orchestrates Amazon machine learning capabilities including: Amazon Translate, Amazon Comprehend, and Amazon Rekognition.
Step 4 - Integration
The inference data integrates with the storage components through an event-driven architecture using EventBridge. EventBridge allows further customization to add additional targets by configuring rules.
Step 5 - Storage and visualization
A combination of Amazon Kinesis Data Firehose, S3 buckets, AWS Glue tables, Amazon Athena, and Amazon QuickSight provide storage and visualization.
- Publish Date
Related content
This blog post teaches readers how to use the Discovering Hot Topics Using Machine Learning solution to draw insights from social media feeds to take advantage of rapidly emerging growth opportunities, address negative sentiment, and improve customer satisfaction. As an illustration, we walk through a business use case in the Media & Entertainment industry.