Category: Analytics

When you deploy your Amazon Elasticsearch Service (Amazon ES) domain to support a production workload, you must choose the type and number of data instances to use, the number of Availability Zones, and whether to use dedicated master instances or not. To follow all the best practice recommendations, you must configure the following: Three dedicated […]

This is a guest post by James Byrne, Engineering Leader at Zendesk, focusing on data pipeline development and operations for the Zendesk Explore analytics product, and Giedrius Praspaliauskas, AWS Solutions Architect. Zendesk is a CRM company that builds support, sales, and customer engagement software designed to foster better customer relationships. From large enterprises to startups, […]

Kibana is an open-source data visualization and exploration tool. It can be used for log and time-series analytics, application monitoring, and operational intelligence use cases. Kibana also offers powerful, easy-to-use features such as histograms, line graphs, pie charts, heat maps, and built-in geospatial support. Kibana is tightly integrated with Amazon Elasticsearch Service (Amazon ES), a […]

Amazon RDS supports publishing PostgreSQL logs to Amazon CloudWatch for versions 9.6.6 and above. Aurora PostgreSQL supports publishing logs to CloudWatch Logs for versions 9.6.12 and above, and versions 10.7 and above. By live streaming this data from CloudWatch to Amazon Elasticsearch Service (Amazon ES), you maintain continuous visibility of RDS PostgreSQL DB logs. You can visualize, […]

Machine-generated data powers solutions and causes problems. It’s indispensable for identifying operational issues in today’s modern software applications, yet you need flexible, scalable tools like Amazon Elasticsearch Service to analyze it in real time. This log data is so valuable that you don’t want to remove it from hot storage, but there’s so much of […]

As applications evolve to be more scalable for the web, customers are adopting flexible data structures and database engines for their use cases. Using NoSQL data stores has become increasing popular because of NoSQL’s flexible data model for building modern applications. Amazon DynamoDB is a fast and flexible NoSQL database service that can provide consistent […]

If you’re building a data lake in the AWS cloud, you’ll most likely want to have metadata and catalog search capability for the underlying data. We recommend Amazon Elasticsearch Service (Amazon ES) for storing and searching S3 keys, and S3 and object metadata. At minimum, your S3 keys include an object name, but they probably […]

A common pattern for web application architecture includes a database for application data coupled with a search engine for searching that data. Many relational and even nonrelational databases offer rudimentary search capabilities. However, search engines add true, complex, natural-language search with relevance and Boolean expressions to other database offerings. As Werner Vogels pointed out recently, […]

When you use Amazon Elasticsearch Service (Amazon ES) for log data, you’re drinking from what usually becomes a forceful firehose. As your Elasticsearch and Kibana knowledge deepens, you find many compelling uses of your data. As your customer base scales up and you scale your infrastructure to handle it, you generate even more log data. […]

Today, Amazon Elasticsearch Service (Amazon ES) announced support for deploying your domains across three Availability Zones (AZ). This feature is available in all AWS Regions that support at least three Availability Zones. With this new feature, you can spread out your master and data nodes to gain better tolerance for Availability Zone failures. Additionally, the AWS Management […]