External reviews are not included in the AWS star rating for the product.
Elastic Search use cases for us involve maintaining a huge amount of data per day, around millions of transactions for each record. We are maintaining all this data with Elastic, and Elastic is doing a fantastic job by doing the indexing. The algorithm is very good, enabling us to process the data very fast.
We are conducting searches with Elastic Search because the data volume is too high. With a couple of indexing configurations, we are able to achieve our goal.
A good feature of Elastic Search is that they have something called policies, which we can make hot and cold, all related to data retention, and that is what I appreciate the most.
From the UI point of view, we are using most probably Kibana, and I think they can do much better than that. That is something they can fine-tune a little bit, and then it will definitely be a good product.
Maintenance in terms of Elastic is that they can improve the UI and UX, and if they fine-tune it a little bit, then it will be much better.
I have used Elastic Search for the last two years in my career.
So far I haven't noticed any lagging, crashing, or downtime with Elastic Search.
The scalability of Elastic Search is good, and I am satisfied with that as of now, and the performance is good.
I don't think I have ever had to contact technical support.
Negative
I find the initial deployment of Elastic Search easy; it is quite straightforward.
Approximately, I am able to deploy Elastic Search within two to three hours for the first time.
To deploy, one or two people will be enough because you need Logstash to be configured to bring the data to Elastic Search for indexing.
We tried to implement big data pipelines and all, and we tried to use Spark as well for analytics and data cleaning, but I think Elastic is better in that field. I didn't find anything better than that.
I am using Elastic Search not only for search purposes but for rendering on maps as well.
I have not searched any vectors so far, so I cannot provide you with the exact output of that.
I was not using vectors in Elastic Search because I was using a vector database. As I mentioned, I use other databases for that. I have not explored it because when it comes to the data, Elastic Search will become expensive. In that case, what I suggest to my clients is to go with PostgreSQL, a vector database, or any other vector database. They are a startup, which is the problem.
We are using streams.
My favorite feature is always aggregations and aggregators. You do not have to do multiple queries and it is always optimized for me.
I always got the perfect results because I am using full text search with aliases and keyword search, everything I am performing it. It always performs out of the box.
It is easy because I have been doing it for years. The last version I remember is 3.5 or 3.1 that I used. Since then, I have been following Elastic Search and the changes they do. For configuration, I have never seen any problem.
Elastic Search consumes lots of memory. You have to provide the heap size a lot if you want the best out of it. The major problem is when a company wants to use Elastic Search but it is at a startup stage. At a startup stage, there is a lot of funds to consider. However, their use case is that they have to use a pretty significant amount of data. For that, it is very expensive. For example, if you take OLTP-based databases in the current scenario, such as ClickHouse or Iceberg, you can do it on 4GB RAM also. Elastic Search is for analytical records. You have to do the analytics on it. According to me, as far as I have seen, people will start moving from Elastic Search sooner or later. Why? Because it is expensive. Another thing is that there is an open source available for that, such as ClickHouse. Around 2014 and 2012, there was only one competitor at that time, which was Solr. But now, not only is Solr there, but you can take ClickHouse and you have Iceberg also. How are we going to compete with them? There is also a fork of Elastic Search that is OpenSearch. As far as I have seen in lots of articles I am reading, users are using it as the ELK stack for logs and analyzing logs. That is not the exact use case. It can do more than that if used correctly. But as it involves lots of cost, people are shifting from Elastic Search to other sources.
When I am talking about pricing, it is not only the server pricing. It is the amount of memory it is using. The pricing is basically the heap Java, which is taking memory. That is the major problem happening here. If we have to run an MVP, a client comes to me and says, "Anurag, we need to do a proof of concept. Can we do it if I can pay a 4GB or 16GB expense?" How can I suggest to them that a minimum of 16GB is needed for Elastic Search so that your proof of concept will be proved? In that case, what I have to suggest from the beginning is to go with Cassandra or at the initial stage, go with PostgreSQL. The problem is the memory it is taking. That is the only thing.
I have been using Elastic Search since around 2012.
I have never seen any instabilities, even from the initial state.
I have checked it for a petabyte of records. It is scalable.
One person can do it, but when it comes to DevOps, we need a team always. Only if we have to manage Elastic Search, one person is fine.
Neutral
I have used Solr and MongoDB as direct alternatives. According to the situation, it basically happens based on what the client wants. Sometimes they want Cassandra in place of Elastic Search. Our thing is only to suggest them. When it comes to the server costing, they are always asking, "Can we move to another server?" For example, I was working with a lower attorney's application and we implemented Elastic Search. For AWS only, we had to take two instances of 32GB for Elastic Search. After a few months only, the client asked, "Anurag, is it possible if we can go to another source if the latency is reduced or if some concurrency will reduce?" In that case, we had to move to Cassandra. Alternatives, I do use them.
Elastic Search is working fine with streaming. I do not have any problem with that. I do not feel any problem with it because the library works well for the solution I am providing in Go. The libraries are healthy over there and it has worked well. I am satisfied with that. If there are some lags, I manage that. I have not used it. My review rating for Elastic Search is 9.5 out of 10.
My use case has evolved over time with Elastic Search. Initially, we started with it as a searching solution. Before Elastic Search, our primary source of truth was SQL databases, the traditional RDBMS. We thought about taking the data from the traditional RDBMS because they were not able to cater to the scale that we wanted to achieve, so we migrated the data from MySQL, keeping it as the primary source of truth, but for the searching mechanism and wildcard searches, we migrated to Elastic Search.
My experience with the relevancy of search results in Elastic Search includes both traditional keywords and full-text search. In the supply chain industry, with millions of orders and customers such as CMA CGM, Maersk, or Kuehne+Nagel, filtering out those orders was essential, using a shipment number, transportation order number, or an origin or destination number. In the gaming industry at FDJ United, full-text searches make more sense to understand gaming intent. For example, when a user searches for 'I really want to play action games', we break down that full-text query, use custom text analyzers, and derive the intent behind the user's query in combination with a vector database alongside Elastic Search.
My assessment of the effectiveness of hybrid search, combining vector and text searches, shows that Elastic Search is remarkable for text-based searches. I have explored other solutions, but none can beat Elastic Search in that area. When I combine hybrid searches with vector databases, they store the mathematical representation of the data. For instance, to find the top 10 closest proximity based on a query, the vector database uses cosine similarity on the available data and suggests the top 10 results while Elastic Search can keep the metadata, enabling quick access to the entire database based on derived intent.
I have utilized trusted GenAI experiences related to semantic search and text-based search in my current project using Elastic Search. My go-to solution for text-based searches will always be Elastic Search, but for semantic search, I am trying to build a solution that emphasizes system-level understanding agents. For example, if a new engineer queries the agent for a system explanation, it scans all the relevant data and provides a comprehensive analysis of the service, contextualizing inputs to reduce hallucination, controlled temperatures for the LLM model, and reducing nucleus sampling. As for knowledge preservation, I use a vector database to store significant outputs generated by the LLM, depending on user preferences regarding the gravity of the analyses performed.
The best features of Elastic Search that I appreciate include its capability for eventual consistent systems where you do not need hard consistency, and it scales very smoothly. For wildcard searches and regex patterns, it really scales massively. It offers ILM, indexation lifecycle management, which allows you to enable a search for a span of six months for the data fed into the system while moving the rest to a new cluster. The structure of the inverted index document facilitates its core features, and I find how Elastic Search understands, indexes, and creates mappings for your data to be remarkable.
While Elastic Search is a good product, I see areas for improvement, particularly regarding the misconception that any amount of data can simply be dumped into Elastic Search. When creating an index, careful consideration of data massaging is essential. Elastic Search stores mappings for various data types, which must remain below a certain threshold to maintain functionality. Users need to throttle the number of fields for searching to avoid overloading the system and ensure that the design of the document is efficient for the Elastic Search index. Additionally, I suggest utilizing ILM periodically throughout the year to manage data shuffling between clusters, preventing hotspots in the distribution of requests across nodes.
I have been using Elastic Search for more than six years.
In terms of stability, I would rate it eight out of ten regarding downtime, bugs, and glitches.
For scalability, I assign it a ten out of ten.
I would rate Elastic Search's technical support as nine out of ten.
Positive
Before Elastic Search, our primary source of truth was SQL databases, the traditional RDBMS.
Estimating the return on investment from Elastic Search is nuanced; however, I can share that initially, search times from traditional RDBMS were around two to three seconds, and with Elastic Search, we reduced that to 50 milliseconds, indicating a significant improvement.
Assessing the complexity of deploying Elastic Search, I have a gray area because a separate DevOps team handles that aspect, but from my experience writing code and utilizing its features, I find it not complex at all.
Estimating the return on investment from Elastic Search is nuanced; however, I can share that initially, search times from traditional RDBMS were around two to three seconds, and with Elastic Search, we reduced that to 50 milliseconds, indicating a significant improvement.
On the subject of pricing, Elastic Search is very cost-efficient. You can host it on-premises, which would incur zero cost, or take it as a SaaS-based service, where the expenses remain minimal.
When comparing Elastic Search to other vendors and products, I have recently explored Algolia, which is also a fully managed service. Elastic Search offers a choice between hosting on-premises or as a fully managed service, which has been beneficial compared to other solutions.
In my company's relationship with the vendor, I have always worked in product-based companies using Elastic Search, often as part of solutions from companies such as Manhattan Associates and in the gaming sector. For B2B industries, they sold to large clients such as Maersk and CMA CGM while my current company, Agoda, operates in the B2C space.
Elastic Search does require some maintenance, especially when considering features such as ILM if you want to enjoy its capabilities. Maintenance tasks depend on the established data pipeline and may introduce some friction.
Currently, we are not using Elastic streams for log ingestion; previously, we utilized the ELK and EFK stacks with Logstash for log ingestion and Kibana for visualization. I also observe a trend where companies migrate to Grafana Loki instead of ELK.
Regarding integration aspects, Elastic Search has exposed REST APIs for all its services, making it easy to integrate with third-party models or endpoints regardless of the underlying infrastructure, as any modern development language can interact with these REST services.
I have not used the attack discovery feature.
My deployment of Elastic Search is on-premises.
At Agoda, we handle over 1.2 billion searches daily, facilitated by Elastic Search.
While I have been at my current company for four months, I am still getting to know my colleagues; however, I know there is a dedicated team focused on Elastic Search. This team exposes a service that acts as an intermediary for communication between Elastic Search and other services.
In my department, there are more than 100 people, whereas the overall organization consists of thousands, exceeding 10,000.
I would rate this review overall as a nine out of ten.
Elastic Cloud (Elasticsearch Service) is a wonderful solution for seamless implementation and maintaining its health. It is much more reliable in the cloud than the on-premises issues that occur very frequently on-premises. However, Elastic does not cover the whole world, and in my region, the Middle East, there are very few hosting places for Elastic Cloud (Elasticsearch Service). It is good news that Elastic recently invested in hosting Elastic Cloud (Elasticsearch Service) in Saudi Arabia, set to launch in March, which I anticipate will lead to more customers adopting Elastic Cloud (Elasticsearch Service) in the very near future.
The only way to visualize data in Elastic, whether it is on-premises or in the cloud, is using Kibana. Kibana's cloud version is not different from the on-premises version, but Elastic Cloud (Elasticsearch Service) is usually more up-to-date, as Elastic maintains and consistently updates Elastic Cloud (Elasticsearch Service) to the latest version, while on-premises versions may lag behind.
I assess the machine learning capabilities of Elastic Cloud (Elasticsearch Service) as truly exceptional, although it is the least used and least understood among many customers. There are quick features that customers can benefit from, such as anomaly detection, but they can also add their own models, which some customers perceive as complex because they do not understand machine learning models and need to have data scientists on their teams to utilize that capability. If a customer uses machine learning in Elastic Cloud (Elasticsearch Service) heavily, they will find that it is very fast to get results compared to using other tools.
Machine learning might be expensive for customers. Customers take advantage of Elastic being open source, but machine learning is not available in the open source version. If a customer is using the open source version without paying licenses to Elastic, they will not enjoy the machine learning features. That is why machine learning does not have the same popularity as Kibana and the other components in Elastic, because only those who pay for Elastic can experience it.
Regarding additional features I would appreciate seeing in the next release of Elastic Cloud (Elasticsearch Service), Elastic acquired Gena AI, and I would appreciate seeing more AI models embedded in the upcoming new versions of Elastic Cloud (Elasticsearch Service). This is what I will be waiting for.
I would rate overall Elastic technical support a seven. It is very noticeable that they are good and responsive, but they heavily collect a lot of logs from customers before resolving issues, which makes the support ticket take longer than expected.
Positive
Some of my customers utilize Elastic Cloud (Elasticsearch Service), especially in the private sector, but most of the government sector do not use it.
Elastic Cloud (Elasticsearch Service) performs well. There are two types of Elastic Cloud (Elasticsearch Service): hosted Elastic Cloud (Elasticsearch Service) and Serverless Elastic Cloud (Elasticsearch Service). Serverless is more expensive compared to hosted Elastic Cloud (Elasticsearch Service), and controlling your bills in serverless sometimes becomes unpredictable, more often than in hosted Elastic Cloud (Elasticsearch Service). Hosted Elastic Cloud (Elasticsearch Service) is not adaptive; it does not rely on data rates, and you will know your spending from day one until the end of the year because unless you change the size of Elastic Cloud (Elasticsearch Service). As long as the size of Elastic Cloud (Elasticsearch Service) is constant, your bill is constant. With serverless, the bill changes frequently based on the influx of the data rate.
I assess Elastic Cloud (Elasticsearch Service)'s ability to handle diverse data sources such as logs and metrics as very good. Elastic managed to unify their data collection through Elastic Agent, the new version of Beats, allowing you to collect various types of data with the same agent. Elastic Cloud (Elasticsearch Service) is performing well in this area, although some data still needs to be ingested by Logstash, but Elastic Agent keeps improving over time.
My overall rating for this product is nine.
