Capture consumer sentiment through social media analytics at scale
This Guidance demonstrates how to design an intelligent brand system that is capable of evaluating social media posts, so you can measure brand performance, create dynamic content, and protect your brand. Amazon SageMaker provides a fully managed infrastructure and tools to build, train, and deploy machine learning (ML) models. Here, pretrained, open-source ML models are used to extract information from social media posts, including consumer sentiment and embeddings from text and images. Next, your team can create a prompt catalog with styles that adhere to your own brand guidelines. When your consumer submits a request for a social media post, the input is processed and a prompt is retrieved from the prompt catalog, with the conversation history securely stored. Finally, the custom prompt is used to generate a selection of personalized posts. Each post receives a predicted engagement rating, and your team can choose the post with the highest engagement rating to publish on social media.
Please note: [Disclaimer]
Architecture Diagram
[Architecture diagram description]
Step 1
Developers ingest social media historical data through the social media’s API and store them in an Amazon Simple Storage Service (Amazon S3) bucket.
Step 2
Amazon SageMaker Jumpstart models are used to enrich the raw data by extracting topic information, post sentiment, and embeddings from text and images.
Step 3
Text embeddings are stored in a vector database to enable searchability through Retrieval Augmented Generation (RAG). The vector database options include Amazon Kendra, Amazon OpenSearch Service, or Amazon RDS for PostgreSQL with pgvector.
Step 4
Data scientists process SageMaker pipelines, which use historical data, embeddings, and engineered features to build a machine learning (ML) model to predict a user post’s engagement rate.
Step 5
Best performing ML models are automatically saved on the SageMaker model registry.
Step 6
Enterprise users create a prompt catalog with different styles to choose from and brand guidelines to be enforced. The prompt catalog is stored in Amazon DynamoDB.
Step 7
The consumer utilizes an app to generate a request for a social media post by providing a description and selecting a specific style (equivalent to a prompt) that aligns with brand guidelines.
The consumer app invokes AWS Lambda, which contains a LangChain orchestrator accessed through Amazon API Gateway. This Lambda function processes the input request and prepares the prompt retrieved from the prompt catalog, while the conversation history is also stored in DynamoDB for future requests.
Step 8
The LangChain orchestrator Lambda function uses RAG to identify similar posts within the historical post data that is stored in vector-embedding databases, and retrieves the most relevant posts. The retrieved relevant posts are used to create a custom prompt together with a post description and the desired style.
Step 9
The LangChain orchestrator uses the custom prompt and invokes Amazon Bedrock and SageMaker JumpStart foundation models (FMs) for text and image generation to generate a selection of personalized posts.
Step 10
Each draft will receive a predicted engagement rating using the model trained earlier from SageMaker. User chooses the draft having highest engagement rate and publishes to social media.
Step 1
Developers ingest social media historical data through the social media’s API and store them in an Amazon Simple Storage Service (Amazon S3) bucket.
Step 2
Amazon SageMaker Jumpstart models are used to enrich the raw data by extracting topic information, post sentiment, and embeddings from text and images.
Step 3
Text embeddings are stored in a vector database to enable searchability through Retrieval Augmented Generation (RAG). The vector database options include Amazon Kendra, Amazon OpenSearch Service, or Amazon RDS for PostgreSQL with pgvector.
Step 4
Data scientists process SageMaker pipelines, which use historical data, embeddings, and engineered features to build a machine learning (ML) model to predict a user post’s engagement rate.
Step 5
Best performing ML models are automatically saved on the SageMaker model registry.
Step 6
Enterprise users create a prompt catalog with different styles to choose from and brand guidelines to be enforced. The prompt catalog is stored in Amazon DynamoDB.
Step 7
The consumer utilizes an app to generate a request for a social media post by providing a description and selecting a specific style (equivalent to a prompt) that aligns with brand guidelines.
The consumer app invokes AWS Lambda, which contains a LangChain orchestrator accessed through Amazon API Gateway. This Lambda function processes the input request and prepares the prompt retrieved from the prompt catalog, while the conversation history is also stored in DynamoDB for future requests.
Step 8
The LangChain orchestrator Lambda function uses RAG to identify similar posts within the historical post data that is stored in vector-embedding databases, and retrieves the most relevant posts. The retrieved relevant posts are used to create a custom prompt together with a post description and the desired style.
Step 9
The LangChain orchestrator uses the custom prompt and invokes Amazon Bedrock and SageMaker JumpStart foundation models (FMs) for text and image generation to generate a selection of personalized posts.
Step 10
Each draft will receive a predicted engagement rating using the model trained earlier from SageMaker. User chooses the draft having highest engagement rate and publishes to social media.
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational ExcellenceRead the Operational Excellence whitepaper
Lambda is a main component of this Guidance, which uses a LangChain orchestrator to effectively interact with multiple LLMs through foundation models hosted on Amazon Bedrock and SageMaker. Lambda, being serverless, removes the need to provision and manage servers; and, API Gateway integrates seamlessly with Lambda, removing operation overhead in running API servers. Also, Amazon CloudWatch offers comprehensive monitoring and near real-time insights into your system's performance, enabling proactive issue detection and rapid response through alarms. Together, Lambda and CloudWatch streamline operations, enhance system resilience, and provide critical observability.
-
SecurityRead the Security whitepaper
AWS Identity and Access Management (IAM), Lambda, Amazon Bedrock, OpenSearch Service, SageMaker, and Amazon S3 collectively help you implement security best practices, like encryption, isolation, and least privilege access. Specifically, Bedrock capabilities include encryption, access control, and it provides secure access to LLMs without sharing the data to LLM providers. OpenSearch Service connects securely to other apps through a Virtual Private Cloud (VPC), security groups, or IP-based policies. Moreover, SageMaker allows for running in VPC mode for network isolation where the access control can be put in place for SageMaker notebooks, endpoints, and other resources through VPC security groups and network access control lists (ACLs). Also, Server-Side Encryption (SSE) for Amazon S3 ensures that the historical posts stored in Amazon S3 are encrypted at rest. Finally, another security safeguard considered for this Guidance include the LangChain orchestrator, which runs in an isolated runtime environment, limiting exposure of your infrastructure and preventing security breaches.
-
ReliabilityRead the Reliability whitepaper
Lambda, DynamoDB, SageMaker, OpenSearch Service, and API Gateway, and were selected for this solution to help your workloads perform their intended functions correctly, consistently, and recover quickly from failure. First, by configuring Lambda, your workloads can automatically scale and manage the processing of code in response to events, ensuring your applications can handle varying workloads without user intervention. Second, DynamoDB offers high availability, automatic backups, and robust data replication, reducing the risk of data loss and ensuring consistent performance, contributing to overall application reliability. Third, the SageMaker inference autoscaling endpoint ensures system reliability by automatically adjusting the number of deployed instances to handle varying workloads, preventing overload and ensuring consistent response times for users. Fourth, OpenSearch Service offers automated backups and automated scaling features, both of which enhance your system's reliability by ensuring data availability and scalability, ultimately contributing to a more dependable system. Finally, API Gateway is resilient to failures with auto-scaling, health checks, and retries; it also supports deployments across multiple Regions.
-
Performance EfficiencyRead the Performance Efficiency whitepaper
The services selected for this Guidance help you monitor the performance of your cloud workloads while allowing you to maintain efficiency as your business needs evolve. For instance, Amazon S3 Intelligent-Tiering provides cost savings by automatically moving data to a lower-cost storage class when data access frequency decreases, optimizing performance and cost-efficiency. Lambda is another service designed to help you get the most out of your cloud workloads. Its configuration is optimized for memory and other settings to help ensure you allocate the right number of resources to functions, thus improving processing efficiency. Cluster management for OpenSearch Service enhances performance by efficiently distributing the workload across nodes in a cluster, ensuring responsive and reliable search capabilities. DynamoDB read and write units allow you to provision and adjust capacity according to your needs, optimizing database performance and cost efficiency. Amazon Kendra can be optimized for performance by tuning its search indexes and relevance rankings, resulting in faster and more accurate search results, improving the efficiency of information retrieval for you and your users.
-
Cost OptimizationRead the Cost Optimization whitepaper
Selecting the appropriate database service, such as Amazon Relational Database Service (Amazon RDS) or Amazon Aurora, can provide cost savings by aligning database capabilities with actual requirements. Also, when using Amazon Bedrock, reducing the prompt length and limiting response token output helps lower computation costs by reducing the workload on language models. Moreover, Amazon Bedrock model adjustments and AWS Inferentia instances contribute to more efficient resource utilization, potentially reducing operational costs while maintaining or even improving performance.
-
SustainabilityRead the Sustainability whitepaper
Lambda, DynamoDB, and Amazon Bedrock can all minimize the environmental impacts of running cloud workloads. Lambda does this by efficiently managing code processing and reducing unnecessary resource consumption and energy use. DynamoDB enhances sustainability by offering automatic backups and robust data replication, reducing the risk of data loss and minimizing the need for additional energy-intensive infrastructure. Additionally, Amazon Bedrock shares the underlying infrastructure with multiple customers, achieving higher energy efficiency as more users utilize the service, reducing the overall environmental impact.
Implementation Resources
A detailed guide is provided to experiment and use within your AWS account. Each stage of building the Guidance, including deployment, usage, and cleanup, is examined to prepare it for deployment.
The sample code is a starting point. It is industry validated, prescriptive but not definitive, and a peek under the hood to help you begin.
Related Content
[Title]
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.