Enable smart text analytics using Amazon OpenSearch Service and Amazon Comprehend

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details.

We’re excited to announce an end-to-end solution that leverages natural language processing to analyze and visualize unstructured text in your Amazon OpenSearch Service domain with Amazon Comprehend in the AWS Cloud. You can deploy this solution in minutes with an AWS CloudFormation template and visualize your data in a Kibana dashboard.

Amazon OpenSearch Service is a fully managed service that delivers Elasticsearch’s easy-to-use APIs and real-time capabilities along with the availability, scalability, and security required by production workloads. Amazon Comprehend is a fully managed natural language processing (NLP) service that enables text analytics to extract insights from the content of documents. Customers can now leverage Amazon OpenSearch and Amazon Comprehend to index and analyze unstructured text, and deploy a pre-configured Kibana dashboard to visualize extracted entities, key phrases, syntax, and sentiment from their documents.

As an example, a company might have large volumes of online customer feedback or transcribed customer calls. With this solution, you can visualize a time series of the sentiment of customer contacts, analyze a word cloud of the entities or key phrases in those contacts, search contacts for a specific product by sentiment, and much more. In this blog post, let’s look at an example Kibana dashboard that you can deploy to draw insights from your text data with Amazon ES and Amazon Comprehend. For detailed instructions, please visit the solution implementation guide.

This solution uses AWS CloudFormation to automate the deployment on the AWS Cloud. You can learn more about the solution by clicking this link and download the template here:

You can use this template to launch the solution and all associated components. Deploying this solution with the default parameters builds the following environment in the AWS Cloud.

The default configuration deploys Amazon API Gateway, AWS Lambda, Amazon OpenSearch Service, and AWS Identity and Access Management roles and policies, but you can also customize the template based on your specific network needs. Once the solution is deployed, you get a fully compatible Amazon ES RESTful API that you can use to ingest documents to Amazon ES and automatically tag the documents with NLP-based text analytics from Amazon Comprehend. You can then use the pre-configured Kibana dashboard to visualize these insights. In the example below, the entity dashboard below shows the word cloud for commercial items, organizations, people, locations, events, and titles from news content.

The sentiment dashboard below shows the sentiment over time, total counts of each sentiment and the top documents with positive and negative sentiment from unstructured text.

The Kibana dashboard is interactive and user-friendly, allowing you to dive deep into your unstructured text data. Try this solution now:

• AWS Solution overview
• Implementation guide
• AWS CloudFormation template

This solution is available in all Regions where Amazon ES and Amazon Comprehend is available. Please refer to the AWS Region Table for more information about Amazon OpenSearch Service and Amazon Comprehend availability.

About the Author

Sameer Karnik is a Sr. Product Manager leading product for Amazon Comprehend, AWS’s natural language processing service.