What does this AWS Solution do?
The Discovering Hot Topics Using Machine Learning solution identifies the most dominant topics associated with your products, policies, events, and brands. This helps you to react quickly to new growth opportunities, address negative brand associations, and deliver a higher level of customer satisfaction for your business. In addition to helping you understand what your customers are saying about your brand, this solution gives you insights into topics that are relevant to your business.
This solution deploys an AWS CloudFormation template to automate data ingestion from these sources:
- RSS news feeds
- YouTube comments tied to videos
- Reddit (comments from subreddits of interest)
- Custom data in JSON or XLSX format
Using the Reddit API, the solution ingests comments from subreddits of interest. The input parameters to the solution includes a list of subreddits to follow for new comments. The comments then undergo NLP analysis to build the Amazon QuickSight dashboard.
Secure one-click deployment
Near real-time analytics
Multi-lingual data ingestion
Pre-built QuickSight dashboard
AWS Solution overview
The diagram below presents the serverless architecture you can automatically deploy using the AWS Solution's implementation guide and accompanying AWS CloudFormation template.
Discovering Hot Topics Using Machine Learning solution architecture
The AWS CloudFormation template automatically deploys AWS Lambda functions, Amazon Simple Storage Service (Amazon S3) buckets, Amazon Kinesis Data Streams, Amazon Simple Queue Service (Amazon SQS) dead-letter-queue (DLQ), Amazon Kinesis Data Firehose, AWS Step Functions workflows, AWS Glue tables, and Amazon QuickSight resources in your account.
The architecture of the solution includes the following key components and workflows:
1. Ingestion – Social media and RSS feed ingestion and management using Lambda functions, Amazon DynamoDB, and Amazon EventBridge. For detailed reference architecture diagrams for Twitter, YouTube comments, RSS news feeds, and custom ingestion using an Amazon S3 bucket, refer to the implementation guide.
2. Data stream – The data is buffered through Amazon Kinesis Data Streams to provide resiliency and throttle incoming requests. The Data Streams have a configured DLQ to catch any errors in processing feeds.
3. Workflow – Consumer (Lambda function) of the Kinesis Data Streams initiates a Step Functions workflow that orchestrates Amazon Machine Learning capabilities including: Amazon Translate, Amazon Comprehend, and Amazon Rekognition.
4. Integration – The inference data integrates with the storage components through an event-driven architecture using Amazon EventBridge. EventBridge allows further customization to add additional targets by configuring rules.
5. Storage and visualization – A combination of Kinesis Data Firehose, Amazon S3 buckets, AWS Glue tables, Amazon Athena, and Amazon QuickSight.
These components are built using the AWS Well-Architected Framework, and the AWS Well-Architected Pillars of Operational excellence, Security, Reliability, Performance efficiency, and Cost optimization—ensuring secure, high-performing, resilient, and efficient infrastructure.
Note: To subscribe to RSS updates, you must have an RSS plug-in enabled for the browser you are using.
This blog post teaches readers how to use the Discovering Hot Topics Using Machine Learning solution to draw insights from social media feeds in order to take advantage of rapidly emerging growth opportunities, to address negative sentiment, and to improve customer satisfaction. As an illustration, we walk through a business use case in the Media & Entertainment industry.
Browse our library of AWS Solutions Implementations to get answers to common architectural problems.
Find AWS certified consulting and technology partners to help you get started.
Browse our portfolio of Consulting Offers to get AWS-vetted help with solution deployment.