The main use cases are for logging, centralized logging system, and security purposes. We also use it for application monitoring and APM to monitor all the applications that run in our environment.

Applications developed by some of our users are monitored using APM, which is one of our primary implementations. For security purposes, we centralize logging for all 6,000 servers using Elastic Search. With more than 12,000 servers in our infrastructure, we need to track which server requires attention and receive alerts. For example, if we need to update all servers, some may be missed, but the system will trigger an alert to notify us. Monitoring and logging are the main functions we use in our current systems.

We are using Elastic Search for log ingestion only.

I chose Elastic Search because it has high search capabilities and setting up the cluster and maintaining it is very easy. Due to this, I found it very user-friendly. High availability and shards allocation are significant advantages that led us to shift to Elastic Search.

I particularly appreciate the sharding concepts because data has high availability. The semantic search feature and the new logsDB feature are valuable additions. These are things I appreciate most about the platform.

Semantic search is a very advanced feature that has proven useful for our data in current systems. I am working with Aadhaar, which is a Unique Identification Authentication firm. When we search for name-related terms, the semantic search provides relevant results. I have also implemented semantic features with hospital data, and it has been very useful for multiple cases.

Elastic Search Hybrid Search is an advanced feature that functions as a future vector database. Vectors are the main component of the database. In current systems, it shows only similar data, but with a vector database, we can store all types of data using vectors. Everything in the future will revolve around vectors. All systems are moving from CPUs to GPUs. This is very useful because comparing vector databases will be a more efficient way to store and retrieve data compared to traditional methods.

Pricing is very high compared to other solutions, but given the features they provide, the pricing is acceptable. The licensing part is also decent compared to other features. I have no issues with this because the features they provide are excellent and position us for next-level future capabilities.

Many banks are moving to Elastic Search, and many identification systems are adopting it because the search capability is significantly higher compared to other solutions, and data retrieval is also very efficient. Many industries are transitioning from old solutions like Splunk to Elastic Search. Banking sectors and healthcare sectors are leading this adoption. Many applications use Elastic Search as their backend, such as Zama. Industries are thinking about and adopting Elastic Search technology because of the features it provides.

There are several areas that need improvement. First, while storing data, there are many mapping issues and mapping conflicts that cause Elastic Search to reject the data. We have to develop solutions or significantly change our processes to address mapping conflicts. This is one of the issues that needs to be fixed.

Second, building semantic search requires significant setup and configuration work. If Elastic Search could provide a one-shot, easy-to-use semantic search implementation, many more users would adopt it. Currently, only a few users are using semantic search, but if they brought it with one-shot ease of use, many people could use it easily and create alerts.

Third, Elastic Search Vector Database needs more attention in the market. We need to bring more features about the vector database to make it easier to set up and use. The use cases also need to be brought to market. Additionally, building dashboards in Kibana is challenging. Compared to Grafana, Kibana has very few features and chart options. We need to enhance Kibana to allow very customized dashboards to be built. Kibana needs significant enhancement in this area.

I have been using Elastic Search for five years.

Elastic Search is stable and reliable until you build the cluster for one terabyte. If data reaches one terabyte, it functions well. However, if data exceeds that or reaches a bottleneck, it becomes unstable. If data is at eighty hundred gigabytes or seven hundred gigabytes, which represents seventy to seventy-five percent of the built cluster capacity, it is very stable and reliable. Search latency is very low compared to other solutions like ClickHouse. Stability and reliability are completely dependent on the data volume.

From the scaling perspective, horizontal scaling by adding extra nodes works well when data increases. We can easily add nodes into the cluster and scale horizontally. Vertical scaling is also straightforward where we can increase the size. We can add new nodes and new components very easily.

I have raised ticket sizes with them many times. I feel very supported by their customer service. For P1 tickets, they provide very immediate quick responses and join calls to support and troubleshoot the issue accordingly. They provide solutions very efficiently. Their service is very good.

I have used Splunk and Dynatrace previously.

I have worked with ClickHouse, and there were many issues with indexing while storing data. The approach is different with ClickHouse. I have also worked with Splunk, and it functioned adequately. However, when storing large setups or large amounts of data, Elastic Search capability is superior and is really useful for the end user.

I believe the initial setup for this solution is complex for new members. However, if you are technically strong and understand how Elastic Search systems work, it is very easy. With five years of experience, I have set up many clusters for banking sectors and healthcare sectors. I have built fourteen clusters in production environments with large-scale systems exceeding five terabytes. This will be typical for those who have technical knowledge and can build easily. Those starting without experience can use Elastic Cloud, which offers very easy one-click deployment. They can deploy an Elastic Search cluster with single clicks. Those with technical knowledge can build the cluster themselves, but those without experience can use Elastic Cloud. This is not an issue.

Correlation alerts is a feature I did not get the opportunity to work on. I have only theoretical knowledge but not practical knowledge.

We can use agentless approaches with a script in addition to agent-based approaches. We are building both agentless and agent-based solutions. Both are good. Agent-based approaches for fetching data work well. Both are functioning well.

Discovery is a feature we are using, and it works well. Attack is a feature I did not get the opportunity to try.

Elastic Search is very user-friendly, and we can easily integrate it with third-party models and other AWS S3 buckets. It is very user-friendly for integrating with other third-party tools.

My overall review rating for this solution is ten out of ten.

I use Elastic Search, and from time to time I use it, but most of the time I am a system administrator. I deployed it more than using it. At the beginning, I was a system administrator, responsible for the deployment and maintenance of Elastic Search clusters. For a few years now, I have started to use it more because the end users are rookie users. They need a lot of help to be able to use Elastic Search effectively. I started to be a user approximately five years ago.

Today, at least, we provide a global, unique Elastic Search cluster for the whole company, and all teams store their logs inside, their traces, and their APM traces. Teams use Kibana to display information. We also use Prometheus exporters to collect metrics from the logs. We execute some query DSL over Elastic Search to collect metrics, which will be injected in a time series database like Prometheus. This is the main usage. We store metrics, logs, and APM traces.

The deployment of Elastic Search is excellent. I like Elastic Search very much for that. I say regularly to the team that Elastic is elastic. It is really difficult to break. This was not the case a few years ago when I worked with Elastic Search version one and version two. Starting with version six of Elastic Search, it started to be really strong. Today, in the past, the main issue was about the data and the volume.

At the moment they integrated lifecycle policy for indices, ILM, Index Life Cycle Management. When it was created, additionally to the data stream, it started to be really easy to have all the same index volume. It is really easy to administrate and to balance data between data centers and between data nodes, and to keep the same everywhere. It is very nice. It is my favorite feature of Elastic Search. It is so easy to manage. Also, maybe because we used it for a long time, we started to be comfortable with all the setup and the node type, and how we should manage our cluster to make it resilient. I think it is really easy to maintain comparatively to some other databases.

To be honest, there is only one downside of Elastic Search that makes sense because we use a basic license, which is a free license. We do not have some features available because of the free license. Except for that, I do not have any complaint. It works perfectly. It is pretty easy to administrate and to use. I do not have complaints, to be honest, except the fact that we do not have all features available such as the APM service map or alerting.

We are not able to use a provider like Sentry, Slack, or PagerDuty. We are forced at some point to generate metrics from the logs in order to use our alerting stack in Prometheus, which works. It is an open-source project which allows us to generate alerts to Slack, PagerDuty, and some third-party tools without any license. However, it is not doable with Elastic Search in the open-source version. The alerting part is the most complicated part to manage because of the license.

From time to time we have some JVM, Java Virtual Machine issues with Elastic Search. However, it is more linked to users' requests. From time to time, people ask Elastic Search to search inside one year of logs without a nice query and without any filters. This is clearly not doable and some nodes will crash. This makes sense. However, Elastic Search is really stable when we do not have this kind of request.

Elastic Search is the perfect tool for scalability. You just need to deploy new nodes. They will be able to join and reach the cluster really easily. I appreciate it for that as well because today at VP, we use Terraform to deploy our infrastructure. All Elastic Search nodes are managed through Terraform. If I need to extend my data node or my ingest node or whatever, I just need to deploy new nodes with the same setup, and the node will join my cluster, and it will scale horizontally really easily.

For logs management, I have not used any alternatives or something similar to Elastic Search. For APM as well, there was a plan in the past to try to migrate to Grafana, the Grafana open-source platform for APM traces using Tempo. Tempo is a Grafana Labs project. However, we decided to keep Elastic Search for that, so we do not have any other tool or similar tool to accomplish that.

Maybe just one, it is about error tracking. We can track errors with APM inside an application, and currently we use Sentry, which is not just an error tracking platform, but also about performance management. However, we use it only for error tracking. It is more useful for developers at the beginning of a new project. Most of the time, they prefer to be connected to Sentry more than APM in order to track errors. When the project will be in production, they will be more focused on the performance than the errors. At this moment they will start to use APM, Elastic Search APM more than Sentry. We do not provide any performance indicators. Sentry is also able to manage performance metrics, but we use it only for errors and everything related to performance has been disabled.

I think the pricing of Elastic Search is really, really expensive. The main point is that we do not get any license. I tried in the past, a few times, to contact the Elastic Search team to get a quote, and it was so complicated each time to get a quote because of the volume and the number of nodes. We are a big company at VP, so we have a lot of nodes, more than one hundred. For sure it was so expensive. They tried to tell me about the enterprise mode and about the new license way to manage cost based on CPU and memory usage. It was really expensive because at this moment, we do not use any cloud services. Our Elastic Search cluster is on-premises.

Everything is self-hosted at VP tech, at VP. We do not have any limit. People using AWS or GCP have limits because the volume of data is really expensive in cloud services and cloud platforms. Because we self-hosted everything around our services such as Elastic Search or Sentry, the idea is to let the user be able to store a lot of data and a lot of metrics. We try to train the team to have a good log level. We do not have such limitation in terms of volume. We have a really big cluster, and at the end, the price is so huge. I gave this review a rating of ten out of ten.