Simplifies Data Management, But Upgrade Challenges
What do you like best about the product?
I find managing data in Elasticsearch very easy compared to other databases, as it doesn't require the hectic re-indexing and maintenance that others do. Setting up an ILM policy lets it take care of Elasticsearch growth, and I particularly like the feature that allows managing the hot, warm, and cold phases based on data requirements. The ability to set how data moves from one tier to another and store historical data in snapshots that can be searched from archival is the best feature for me. Also, the initial setup of Elasticsearch was easy, which is a big plus.
What do you dislike about the product?
Elasticsearch upgrade from version to another is always a problem. They don't allow you to jump 2 versions using a rolling upgrade, as any particular version like V1 does not allow you to have any index which was created in V1-2 version.
What problems is the product solving and how is that benefiting you?
I use Elasticsearch for fast search and data archival, storing trading data for 7 years. Managing Elasticsearch is easy with ILM, allowing efficient data tier management without constant re-indexing.
Impressive Speed and Powerful Near Real-Time Search with Elasticsearch
What do you like best about the product?
Elasticsearch delivers impressive search speed and strong performance, even when working with massive datasets. Its near real-time search capability, combined with powerful full-text search features, makes it a key part of our data infrastructure.
What do you dislike about the product?
Elasticsearch can be quite resource-intensive, particularly when it comes to RAM usage. For smaller infrastructure setups, managing JVM heap sizes and making sure the cluster has sufficient memory can quickly become a bit of a headache.
What problems is the product solving and how is that benefiting you?
Elasticsearch solves the problem of searching through massive amounts of unstructured data that traditional SQL databases struggle to handle efficiently. It provides a highly scalable, distributed environment that ensures fast retrieval times.
This benefits me by significantly reducing latency in our application's search feature and providing powerful analytical tools through its aggregation framework. It allows us to monitor logs in real-time and deliver a seamless, Google-like search experience to our end users.
Dynamic queries have boosted search speed and now support flexible unstructured data storage
What is our primary use case?
As a developer, I use Elastic Search in developing one of my applications, basically integrating the back-end with Elastic Search.
Our main use case for Elastic Search is for Logstash, which is a subset of Elastic Search that allows us to store logs and enables searching between logs with specific keywords in specific time ranges. Apart from that, we have our data stored in an index, and since Elastic Search is a NoSQL database, that's how we store the files in our databases.
The main objective of integrating Elastic Search is to transition from normal SQL databases to have faster searches and dynamic queries built around it, which makes the search much quicker. Since not all data is structured, we also need to handle unstructured data, and that's how Elastic Search has replaced our previous system.
How has it helped my organization?
The positive impact I've seen from using Elastic Search includes replacing conventional databases and being able to store much more unstructured data. In the future, if we need to include data not present in earlier systems, we can implement semantic or flyway changes with Elastic Search in place, allowing us to store unstructured data as is.
What is most valuable?
The most valuable feature of Elastic Search that I appreciate is the dynamic query building and the speed of result fetching, especially since we have an open-source version called OpenSearch that we use in specific places due to the cost of storing data with Elastic Search.
Dynamic query building and result fetching are valuable because there are specific use cases where we need to build queries based on environment variables rather than having a generic query. This dynamic building helps address various business scenarios, especially considering customer product types and flags that may need inclusion or exclusion in the query. It allows me to create one query to accommodate multiple business cases and ensures that user-specific scenarios are included, with results already fetched for each.
What needs improvement?
Elastic Search has many features, including Kibana and Logstash, which we regularly use. However, one downside in our product is cost, as it can be expensive when maintaining multiple shards and indexes. Failures of shards or nodes can occur, and I can mention that cost and the upscaling of nodes or shards are areas needing improvement.
We haven't explored the hybrid search feature of Elastic Search, which combines vector and text searches, yet.
Scalability of Elastic Search presents disadvantages, particularly when handling minimal or production-level data. It manages high volumes of unstructured data well, but during performance tests involving one million requests at once, we encountered issues with shards and nodes not upscaling as needed, leading to crashes and minimal data loss, which isn't typical in real-world scenarios.
For how long have I used the solution?
I have been working with Elastic Search for about 1.5 years.
What do I think about the stability of the solution?
Elastic Search is quite reliable for us, and despite identifying some very minute limitations, we still rely on Elastic Search.
What do I think about the scalability of the solution?
Scalability of Elastic Search presents disadvantages, particularly when handling minimal or production-level data. It manages high volumes of unstructured data well, but during performance tests involving one million requests at once, we encountered issues with shards and nodes not upscaling as needed, leading to crashes and minimal data loss, which isn't typical in real-world scenarios.
How are customer service and support?
I have not communicated with the technical support of Elastic Search at all up to this point.
Which solution did I use previously and why did I switch?
Before Elastic Search, we used Couchbase, which is also a NoSQL database. Initially, it was free software integrated into our applications, but with its commercialization, we explored alternatives and found that Elastic Search would be a better fit than Couchbase.
How was the initial setup?
We conducted some preliminary research to find a potential replacement for Couchbase while searching for NoSQL databases. The good documentation for Elastic Search on various websites helped us conclude that it would be an ideal fit. Although we considered the open-source version known as OpenSearch, we decided to integrate Elastic Search to explore its features, eventually determining it had much more powerful features, such as the Kibana dashboard and Logstash.
What was our ROI?
With respect to performance, we have seen a return on investment from Elastic Search. For example, the API response time has improved significantly, cutting the time down from about one or two minutes to around 50% faster, benefiting our downstream applications.
Search through massive message archives in milliseconds and have supported large compliance data
What is our primary use case?
I can describe a few use cases for Elastic Search because in my previous company, we had a message database and needed to implement a search system. We first used Postgres full text search, but it did not work well, so we had to migrate everything into Elastic Search. Elastic Search could better index the data and we could search every document in instant time.
The key differences between Elastic Search and Postgres search, including both pros and cons, are primarily related to indexing speed. In Postgres, the full text search speed is quite noticeable if you have a message document. In Elastic Search, I am not quite certain about when comparing to normal data, but for our use case of searching through message documents, the speed difference is noticeable in Postgres because our documents are very large. Since Elastic Search is primarily built for search, I think it can better search through the document. Our documents were sometimes really large, ranging from 100 megabytes to 200 megabytes per document, so I think Elastic Search handles this much better than Postgres.
What is most valuable?
What I appreciate about Elastic Search is that the best features include the ability to search through very big documents and index and search through them really fast. This is the one thing I value most about Elastic Search.
Regarding stability, I have not had any crashes, downtimes, or performance issues with it. We did have one incident, but it was not from Elastic Search. I think it was an AWS service outage. The downtime was an AWS error, not from Elastic Search.
Concerning scalability, I find it scalable because it is quite scalable right now. We currently have a terabyte of compliance data, and the client can search through that very effectively. We have not experienced any scalability errors so far. I think our compliance data amounts to approximately five or six terabytes of data, which is very large. We can search through that document quite easily, sometimes in 7 milliseconds, sometimes one or two milliseconds. It was quite fast.
What needs improvement?
Apart from the good things, what I would like to see improved or enhanced in Elastic Search is the storage cost. I think the main problem with Elastic Search is that sometimes the storage was quite expensive. We also have a file system in addition to compliance. We have an FDS on our server, and we sometimes want to attach something on top of the FDS and search through every file without having to create a search index dedicatedly.
The missing features or functionalities in Elastic Search that I would like to see included in the future or some functionality that requires enhancement would be the ability to attach to our file system, such as network file system or NFS, or maybe our on-premise NAS server, and then search through everything, whether it is a document, text, or some information from those documents. That may be our primary use case right now, but we do not have that capability. Additionally, I would like to see a better search system so we can locally embed and find through everything.
For how long have I used the solution?
I have been working with Elastic Search for approximately one or two years.
What do I think about the stability of the solution?
Regarding stability, I have not had any crashes, downtimes, or performance issues with it. We did have one incident, but it was not from Elastic Search. I think it was an AWS service outage, not from Elastic Search. The error was an AWS error.
What do I think about the scalability of the solution?
Concerning scalability, I find it scalable because it is quite scalable right now. We currently have a terabyte of compliance data, and the client can search through that very effectively. We do not have any scalability errors so far. I think our compliance data amounts to approximately five or six terabytes of data, which is very large. We can search through that document quite easily, sometimes in 7 milliseconds, sometimes one or two milliseconds. It was quite fast.
How are customer service and support?
I do not know anything about the tech support because I have not escalated any questions to the technical support or customer service teams. We have not talked to anyone.
Which solution did I use previously and why did I switch?
I previously used a different solution for search. The solution I used for the search previously was Postgres full text search.
How was the initial setup?
The initial setup process of Elastic Search was straightforward. I did not face many challenges or complexities except for the fact that we had to extract every document and build a search index. Aside from that, we did not experience much complexity during that time.
What's my experience with pricing, setup cost, and licensing?
When it comes to pricing, I think we had to pay AWS approximately 1,000 to 1,200 per month for the overall stack. I am not quite certain about how much Elastic Search costs specifically because I was not in charge of pricing. The overall system cost was approximately 1,200 to 1,500 per month.
I do not find it cost-effective. I am not quite certain. Maybe the client might complain, but I am not certain. We just built out the system.
Which other solutions did I evaluate?
Before choosing Elastic Search, I evaluated other options. At first, we tried to go with Redis search because we really needed fast retrieval, but Redis search was closed source at that time, so we could not go with Redis search. We had to try Elastic Search and it performed quite surprisingly well.
What other advice do I have?
Given my experience with Elastic Search, a piece of advice or recommendation I may share with other organizations considering it is that if you are looking for a simple search, I am not certain whether I would recommend Elastic Search. However, if you are handling message data with a massive amount of data and you need sub-millisecond search time, I think in that scenario Elastic Search outperforms everything. I would give this product a rating of eight out of ten. Especially if you are using SQL to search through the data, Elastic Search really outperforms SQL when you have to search through massive data.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)
Indexing millions of daily records has been streamlined and search performance meets our needs
What is our primary use case?
Elastic Search use cases for us involve maintaining a huge amount of data per day, around millions of transactions for each record. We are maintaining all this data with Elastic, and Elastic is doing a fantastic job by doing the indexing. The algorithm is very good, enabling us to process the data very fast.
We are conducting searches with Elastic Search because the data volume is too high. With a couple of indexing configurations, we are able to achieve our goal.
What is most valuable?
A good feature of Elastic Search is that they have something called policies, which we can make hot and cold, all related to data retention, and that is what I appreciate the most.
What needs improvement?
From the UI point of view, we are using most probably Kibana, and I think they can do much better than that. That is something they can fine-tune a little bit, and then it will definitely be a good product.
Maintenance in terms of Elastic is that they can improve the UI and UX, and if they fine-tune it a little bit, then it will be much better.
For how long have I used the solution?
I have used Elastic Search for the last two years in my career.
What do I think about the stability of the solution?
So far I haven't noticed any lagging, crashing, or downtime with Elastic Search.
What do I think about the scalability of the solution?
The scalability of Elastic Search is good, and I am satisfied with that as of now, and the performance is good.
How are customer service and support?
I don't think I have ever had to contact technical support.
How was the initial setup?
I find the initial deployment of Elastic Search easy; it is quite straightforward.
Approximately, I am able to deploy Elastic Search within two to three hours for the first time.
What about the implementation team?
To deploy, one or two people will be enough because you need Logstash to be configured to bring the data to Elastic Search for indexing.
Which other solutions did I evaluate?
We tried to implement big data pipelines and all, and we tried to use Spark as well for analytics and data cleaning, but I think Elastic is better in that field. I didn't find anything better than that.
Fast, Scalable Elasticsearch for Quick Log Analysis
What do you like best about the product?
From our use, Elasticsearch is fast, scalable and provides quick results for querying which makes it very useful for any log analysis
What do you dislike about the product?
Operational cost is increasing
Shard allocation and indexing can be made easier to configure
What problems is the product solving and how is that benefiting you?
We use ELK for log parsing, and with it its ability to respond quickly to queries helps us identify issues and get clues about what’s going wrong much faster.
Powerful and Scalable Search Solution
What do you like best about the product?
What I like most about Elasticsearch is its speed and flexibility. It handles large amounts of data efficiently and makes searching very fast. It is also versatile enough to be used for both search and analytics use cases.
What do you dislike about the product?
One thing I dislike about Elasticsearch is that it can become complex to manage as it grows. It requires careful planning and monitoring to avoid performance and stability issues. Licensing and pricing changes over time have also created some uncertainty for users.
What problems is the product solving and how is that benefiting you?
Elasticsearch helps us quickly search and analyze large amounts of data in one place. It makes it easier to find relevant information, monitor systems, and generate insights from logs or application data. This improves visibility and allows us to respond to issues faster and make better decisions.
Powerful Log Database with Helpful Integrations for Easy Parsing
What do you like best about the product?
You can use it as a database and classify all type of logs. The integrations they have helps you to parse them
What do you dislike about the product?
Sometimes correlations can be difficult between different technologies
What problems is the product solving and how is that benefiting you?
Handling logs
Efficient Log Management & Search with Excellent Support
What do you like best about the product?
Very efficient product to manage our logs and search
The support is easy to interact with and the quality of the answers are perfect
What do you dislike about the product?
When it is self managed, a bit tedious to update
What problems is the product solving and how is that benefiting you?
Centralizing our documentation and making it available in quick search is really great
Best No-SQL Databases with vector search and AI use cases
What do you like best about the product?
It’s one of the best NoSQL databases on the market. It makes it easier to collect logs from many different sources and to define integrations for them. It provides many features within one tool like vector search, machine learning, alerting and a lot
What do you dislike about the product?
I don’t like the breaking changes that come with version upgrades, because they have a big impact when multiple teams depend on the deployment.
What problems is the product solving and how is that benefiting you?
We collect telecom metrics from around 1,000 servers, which helps us search for and debug errors, create KPIs, and set up rules and alerting based on that data. As a result, it reduces manual effort and is easy to integrate with other systems. The best part is elasticsearch can be used for varied use cases. Its a single point of monitoring for our whole telecom stack.
