Posted On: Jun 4, 2020
Amazon Elasticsearch Service now offers anomaly detection, which uses machine learning to detect anomalies on real-time streaming data and identifies issues as they evolve so you can mitigate them immediately. This new feature is built on Random Cut Forests (RCF), a proven algorithm for real-time streaming, and is domain agnostic, making it a great choice for a wide range of log analytics applications.
Static, rule-based analytics approaches struggle to adapt to dynamic workloads and are prone to miss critical issues. Amazon Elasticsearch Service anomaly detection leverages RCF, an unsupervised algorithm, that continuously adapts to evolving data patterns. The anomaly detection feature is lightweight and resilient, with the computational load distributed across Elasticsearch nodes, eliminating the need for dedicated machine learning nodes. This efficient design allows the feature to handle large volumes of data without affecting cluster performance or application workloads.
The new anomaly detection feature includes a Kibana user interface that provides context into the data and events that contributed to an anomaly, making it easy for all users, regardless of their machine learning knowledge, to derive value from the feature. You can use anomaly detection with alerting to trigger notifications as outliers are detected.
The real-time anomaly detection feature is powered by Open Distro for Elasticsearch, an Apache 2.0-licensed distribution of Elasticsearch. To learn more about Open Distro for Elasticsearch and its anomaly detection plugin, visit the project website.
Anomaly detection is available on all domains running Elasticsearch 7.4 at no additional cost. To learn more, see the documentation.
The real-time anomaly detection feature for Amazon Elasticsearch Service is now available in 22 regions globally: US East (N. Virginia, Ohio), US West (Oregon, N. California), Canada (Central), South America (Sao Paulo), EU (Ireland, London, Frankfurt, Paris, Stockholm, Milan), Asia Pacific (Singapore, Sydney, Tokyo, Seoul, Mumbai, Hong Kong), Middle East (Bahrain), China (Beijing – operated by Sinnet, Ningxia – operated by NWCD), Africa (Cape Town). Please refer to the AWS Region Table for more information about Amazon Elasticsearch Service availability.