I use Elastic Search, and from time to time I use it, but most of the time I am a system administrator. I deployed it more than using it. At the beginning, I was a system administrator, responsible for the deployment and maintenance of Elastic Search clusters. For a few years now, I have started to use it more because the end users are rookie users. They need a lot of help to be able to use Elastic Search effectively. I started to be a user approximately five years ago.

Today, at least, we provide a global, unique Elastic Search cluster for the whole company, and all teams store their logs inside, their traces, and their APM traces. Teams use Kibana to display information. We also use Prometheus exporters to collect metrics from the logs. We execute some query DSL over Elastic Search to collect metrics, which will be injected in a time series database like Prometheus. This is the main usage. We store metrics, logs, and APM traces.

The deployment of Elastic Search is excellent. I like Elastic Search very much for that. I say regularly to the team that Elastic is elastic. It is really difficult to break. This was not the case a few years ago when I worked with Elastic Search version one and version two. Starting with version six of Elastic Search, it started to be really strong. Today, in the past, the main issue was about the data and the volume.

At the moment they integrated lifecycle policy for indices, ILM, Index Life Cycle Management. When it was created, additionally to the data stream, it started to be really easy to have all the same index volume. It is really easy to administrate and to balance data between data centers and between data nodes, and to keep the same everywhere. It is very nice. It is my favorite feature of Elastic Search. It is so easy to manage. Also, maybe because we used it for a long time, we started to be comfortable with all the setup and the node type, and how we should manage our cluster to make it resilient. I think it is really easy to maintain comparatively to some other databases.

To be honest, there is only one downside of Elastic Search that makes sense because we use a basic license, which is a free license. We do not have some features available because of the free license. Except for that, I do not have any complaint. It works perfectly. It is pretty easy to administrate and to use. I do not have complaints, to be honest, except the fact that we do not have all features available such as the APM service map or alerting.

We are not able to use a provider like Sentry, Slack, or PagerDuty. We are forced at some point to generate metrics from the logs in order to use our alerting stack in Prometheus, which works. It is an open-source project which allows us to generate alerts to Slack, PagerDuty, and some third-party tools without any license. However, it is not doable with Elastic Search in the open-source version. The alerting part is the most complicated part to manage because of the license.

From time to time we have some JVM, Java Virtual Machine issues with Elastic Search. However, it is more linked to users' requests. From time to time, people ask Elastic Search to search inside one year of logs without a nice query and without any filters. This is clearly not doable and some nodes will crash. This makes sense. However, Elastic Search is really stable when we do not have this kind of request.

Elastic Search is the perfect tool for scalability. You just need to deploy new nodes. They will be able to join and reach the cluster really easily. I appreciate it for that as well because today at VP, we use Terraform to deploy our infrastructure. All Elastic Search nodes are managed through Terraform. If I need to extend my data node or my ingest node or whatever, I just need to deploy new nodes with the same setup, and the node will join my cluster, and it will scale horizontally really easily.

For logs management, I have not used any alternatives or something similar to Elastic Search. For APM as well, there was a plan in the past to try to migrate to Grafana, the Grafana open-source platform for APM traces using Tempo. Tempo is a Grafana Labs project. However, we decided to keep Elastic Search for that, so we do not have any other tool or similar tool to accomplish that.

Maybe just one, it is about error tracking. We can track errors with APM inside an application, and currently we use Sentry, which is not just an error tracking platform, but also about performance management. However, we use it only for error tracking. It is more useful for developers at the beginning of a new project. Most of the time, they prefer to be connected to Sentry more than APM in order to track errors. When the project will be in production, they will be more focused on the performance than the errors. At this moment they will start to use APM, Elastic Search APM more than Sentry. We do not provide any performance indicators. Sentry is also able to manage performance metrics, but we use it only for errors and everything related to performance has been disabled.

I think the pricing of Elastic Search is really, really expensive. The main point is that we do not get any license. I tried in the past, a few times, to contact the Elastic Search team to get a quote, and it was so complicated each time to get a quote because of the volume and the number of nodes. We are a big company at VP, so we have a lot of nodes, more than one hundred. For sure it was so expensive. They tried to tell me about the enterprise mode and about the new license way to manage cost based on CPU and memory usage. It was really expensive because at this moment, we do not use any cloud services. Our Elastic Search cluster is on-premises.

Everything is self-hosted at VP tech, at VP. We do not have any limit. People using AWS or GCP have limits because the volume of data is really expensive in cloud services and cloud platforms. Because we self-hosted everything around our services such as Elastic Search or Sentry, the idea is to let the user be able to store a lot of data and a lot of metrics. We try to train the team to have a good log level. We do not have such limitation in terms of volume. We have a really big cluster, and at the end, the price is so huge. I gave this review a rating of ten out of ten.

We utilize Elastic Search to bring a bunch of data sources together into a large search corpus, which is used to power our core research platform.

We don't generally do a lot of full-text search with Elastic Search. We do a lot of keyword-based searching and a lot of faceted search, and it works really well. We've also had to build custom relevance algorithms based on data that's being stored in the search index. This is more about the algorithm being less about text matching and more about feature matching and relevance on a number of different scales. It's generally worked out really well.

The best feature of Elastic Search is it does exactly what it says. It's really easy to get set up and running and have search running very quickly with basic, out-of-the-box features. It scales very well, and we can do a whole lot with the core feature set before having to move to more advanced concepts. Even then, it performs very well, whether we need to expand into vector databases or decide that the Elastic Search Query DSL doesn't solve our needs anymore and have to go with ESQL or something. It expands and scales really well.

The hosted solution means Elastic Search takes care of the maintenance, which is one of the reasons we chose it. There's been very little maintenance from a data perspective on our side. As we make changes to our database structure, we've had to mirror them into Elastic Search.

We haven't had the opportunity to use the hybrid search with Elastic Search yet. I think there's a place for it in our long-term solution, but we're not quite there yet.

We haven't yet used any AI features built into Elastic Search.

To do what we want to do with Elastic Search, the queries can get complex and require a fuller understanding of the DSL. Once we start to build that understanding, it's another muscle we have, so it's not a bad thing, but it just takes a while to get up and running with expertise for our engineers.

It's not hard to learn how to use more complex things in Elastic Search; it's just a challenge we're going to face.

In my career, I've been using Elastic Search for three or four companies, probably on and off for 10 years.

We've had various very small blips with Elastic Search, but it's never been an issue that was concerning. We have limited infrastructure, so we could go further in terms of our hosted deployment to ensure that some of those things didn't happen. We've simply accepted the level of risk we have.

Thus far, everything seems really good in terms of scalability for Elastic Search. We don't have the largest data set in the world; we have millions of records, single-digit millions, so two or three million records. I feel confident knowing that we could times that by 10 or 100, maybe, and it would still work. The cost would obviously scale, the number of nodes would scale, but Elastic Search would be able to handle that level of scale.

Before I was using Elastic Search and actually before Elastic Search even existed, I previously used Apache Solr and Lucene in my career. The release of Elastic Search way back when was a boon because it was out of the box and did what it said. We've also worked with Pinecone, Amazon's OpenSearch, and essentially Postgres trying to do vector search in Postgres. All of those tools have their place, but if we're doing straight search, Elastic Search is just really the right answer.

The initial deployment of Elastic Search was really straightforward because we used the hosted solution.

We had Elastic Search live and our first initial searches running in our staging environment within a week. We moved into production with our full data set within six weeks.

We had one engineer working on this implementation. That's why it took six weeks.

Elastic Search's pricing is affordable when using the hosted solution through Elastic Search. The pay-as-you-go monthly approach has been nice, and if we scale as a company grows, we'll probably switch to a prepaid model, which will be an even bigger benefit. Having the hosted solution and not having to pay for essentially a DevOps person on staff to manage makes it affordable. We haven't really looked into serverless, which has its own benefits. I think serverless still had some challenges early on, and I wanted to go with something I had previously worked with. The hosted solution pricing fits, but the pricing for serverless also looks really interesting. The self-managed solution is nice from a pricing perspective, but we need the right staff to support it, and we don't have that staff.

We don't use Elastic Search for log ingestion, though I think they have a feature for this.

We haven't worked with anything in terms of Elastic Search integration process for third-party models with interference endpoints.

I'm not using the Attack Discovery feature because we're not using Elastic Search for our observability approach.

We have no partnerships or anything with Elastic Search. I would rate this review as a 9.

I developed a chatbot with text summarization and question answering capabilities. I need to summarize multiple PDFs, and I have a database in Google Cloud Storage where I perform keyword matching with Elastic Search using exact keyword matching.

I have different clients, but I use Elastic Cloud (Elasticsearch Service) for one specific client. For that one job, Elastic Cloud (Elasticsearch Service) is the main tool because I am using an Elastic Search strategy instead of a vector database.

Scalability is valuable to me. I have 50,000 PDF JSON files that contain my metadata, and I am really glad to use Elastic Cloud (Elasticsearch Service) for this volume without any issues. From a startup's perspective, I would say that until 10 GB of storage, there is no problem whatsoever.

Application-wise, everything was easy to work with. I really appreciated their dashboard because everything was clear, and it was easy to implement.

Because I am pursuing a PhD and work under the university, my university has an agreement with AWS, which makes it essentially free and easier to use. In the AWS ecosystem, everything is connected and I can control everything without uncertainty about what is happening behind the scenes. However, when using Elastic Cloud (Elasticsearch Service), I connected it to Google Cloud but I am paying separate receipts. Over the last two months in October and November, I paid two separate invoices that are not connected to Google Cloud, which I did not appreciate.

Google Cloud has a nice interface that gives me full control of pricing and billing. I can see daily, weekly, and monthly breakdowns with bar charts, and I can track exactly how much I spent during any period. Elastic Cloud (Elasticsearch Service) does not have such a tool for billing visibility. Since I am handling significant amounts of money and am responsible for this task within my company, I have high expectations for pricing and billing transparency. I would appreciate the ability to set a spending limit, such as uploading 200 euros, and receive notifications when reaching 50% of that limit. These notifications could appear on the dashboard, in the application, or via email. It would be valuable to see a timeline of my spending.

I would characterize the pricing as somewhat expensive. I did not use competitors extensively, so I may have a bias about this. The pricing of large language models is not expensive—I use Anthropic's Claude or Google's Gemini, which are state-of-the-art models. However, I am uncertain whether I have a bias about Elastic Cloud (Elasticsearch Service) pricing. It is not extraordinarily expensive, but when I compare it with the cost of using large language models or Google Cloud storage, it is quite expensive.

A couple of days ago, the Elastic team reached out to me. We have been regularly using the service since April, and 10 days ago at the beginning of December, I deleted my hosted deployments because I did not like the idea of paying when I am not actively using Elastic Cloud (Elasticsearch Service). They informed me that there is a serverless option available. Before Christmas, I want to try it to see how it works, as I am uncertain about the serverless concept and whether it will provide the same functionality that I use with the hosted deployment.

I have been using this since April.

I have experienced no issues whatsoever in the last five or six months. Whenever I perform my searches, and because my application is active with clients using it, there has been no feedback about problems. During my tests, I did not observe any lag or delay.

I would give them a rating of ten. They were extremely helpful, kind, and communicative. Three people from the Elastic team spoke with me, and they were genuinely trying to solve the problem and understand my expectations. It was really excellent, and I recommend them highly.

I used the hosted deployment version, not the serverless option yet. The hosted deployment took me 10 to 15 minutes to set up. It was very easy and primarily involved indexing. I generated Python code on my local computer within minutes and pushed everything with the indexing. It took about 15 to 20 minutes total, though this is related to the size of my folder. I was indexing around 10,000 PDFs and creating metadata JSON files. The process was easy and fast.

I also use AWS daily in their ecosystem, which contains everything I need. I use Google Cloud primarily because of the pricing, as I must consider profit margins.

My team is small, consisting of about four or five people. On Elastic Cloud (Elasticsearch Service), only two of us work with it, but I am the one who uses it daily.

So far, I have not performed any maintenance.

Elastic Cloud (Elasticsearch Service) focuses on exact keyword matching. This means they do not address semantic similarity well. For example, if I use a word and then use another word with the same meaning in a sentence but not a synonym or similar word, Elastic cannot understand this semantic similarity, which is important for my chatbots. This is why I was using vector databases, as they focus on semantic similarity of words and tokens, whereas Elastic looks for exact word representation.

The team mentioned that hybrid search is an option. I have my own vector database that I use daily as a personal solution, and I could give hybrid search a chance, but I have not tried it yet.

I would rate my overall experience as seven out of ten. Two points are deducted for the pricing, which was higher than my expectations. The pricing has been significant over the last two months and was considerably more than I anticipated. My overall review rating for this service is nine out of ten.