ElasticCloud(Elasticsearch, FedRAMP, SaaS Contract) [Private Offer Only]

The main use cases are for logging, centralized logging system, and security purposes. We also use it for application monitoring and APM  to monitor all the applications that run in our environment.

Applications developed by some of our users are monitored using APM , which is one of our primary implementations. For security purposes, we centralize logging for all 6,000 servers using Elastic Search . With more than 12,000 servers in our infrastructure, we need to track which server requires attention and receive alerts. For example, if we need to update all servers, some may be missed, but the system will trigger an alert to notify us. Monitoring and logging are the main functions we use in our current systems.

We are using Elastic Search  for log ingestion only.

I chose Elastic Search because it has high search capabilities and setting up the cluster and maintaining it is very easy. Due to this, I found it very user-friendly. High availability and shards allocation are significant advantages that led us to shift to Elastic Search.

I particularly appreciate the sharding concepts because data has high availability. The semantic search feature and the new logsDB feature are valuable additions. These are things I appreciate most about the platform.

Semantic search is a very advanced feature that has proven useful for our data in current systems. I am working with Aadhaar, which is a Unique Identification Authentication firm. When we search for name-related terms, the semantic search provides relevant results. I have also implemented semantic features with hospital data, and it has been very useful for multiple cases.

Elastic Search Hybrid Search is an advanced feature that functions as a future vector database. Vectors are the main component of the database. In current systems, it shows only similar data, but with a vector database, we can store all types of data using vectors. Everything in the future will revolve around vectors. All systems are moving from CPUs to GPUs. This is very useful because comparing vector databases will be a more efficient way to store and retrieve data compared to traditional methods.

Pricing is very high compared to other solutions, but given the features they provide, the pricing is acceptable. The licensing part is also decent compared to other features. I have no issues with this because the features they provide are excellent and position us for next-level future capabilities.

Many banks are moving to Elastic Search, and many identification systems are adopting it because the search capability is significantly higher compared to other solutions, and data retrieval is also very efficient. Many industries are transitioning from old solutions like Splunk to Elastic Search. Banking sectors and healthcare sectors are leading this adoption. Many applications use Elastic Search as their backend, such as Zama. Industries are thinking about and adopting Elastic Search technology because of the features it provides.

There are several areas that need improvement. First, while storing data, there are many mapping issues and mapping conflicts that cause Elastic Search to reject the data. We have to develop solutions or significantly change our processes to address mapping conflicts. This is one of the issues that needs to be fixed.

Second, building semantic search requires significant setup and configuration work. If Elastic Search could provide a one-shot, easy-to-use semantic search implementation, many more users would adopt it. Currently, only a few users are using semantic search, but if they brought it with one-shot ease of use, many people could use it easily and create alerts.

Third, Elastic Search Vector Database needs more attention in the market. We need to bring more features about the vector database to make it easier to set up and use. The use cases also need to be brought to market. Additionally, building dashboards in Kibana is challenging. Compared to Grafana , Kibana has very few features and chart options. We need to enhance Kibana to allow very customized dashboards to be built. Kibana needs significant enhancement in this area.

I have been using Elastic Search for five years.

Elastic Search is stable and reliable until you build the cluster for one terabyte. If data reaches one terabyte, it functions well. However, if data exceeds that or reaches a bottleneck, it becomes unstable. If data is at eighty hundred gigabytes or seven hundred gigabytes, which represents seventy to seventy-five percent of the built cluster capacity, it is very stable and reliable. Search latency is very low compared to other solutions like ClickHouse . Stability and reliability are completely dependent on the data volume.

From the scaling perspective, horizontal scaling by adding extra nodes works well when data increases. We can easily add nodes into the cluster and scale horizontally. Vertical scaling is also straightforward where we can increase the size. We can add new nodes and new components very easily.

I have raised ticket sizes with them many times. I feel very supported by their customer service. For P1 tickets, they provide very immediate quick responses and join calls to support and troubleshoot the issue accordingly. They provide solutions very efficiently. Their service is very good.

I have used Splunk and Dynatrace  previously.

I have worked with ClickHouse , and there were many issues with indexing while storing data. The approach is different with ClickHouse. I have also worked with Splunk, and it functioned adequately. However, when storing large setups or large amounts of data, Elastic Search capability is superior and is really useful for the end user.

I believe the initial setup for this solution is complex for new members. However, if you are technically strong and understand how Elastic Search systems work, it is very easy. With five years of experience, I have set up many clusters for banking sectors and healthcare sectors. I have built fourteen clusters in production environments with large-scale systems exceeding five terabytes. This will be typical for those who have technical knowledge and can build easily. Those starting without experience can use Elastic Cloud, which offers very easy one-click deployment. They can deploy an Elastic Search cluster with single clicks. Those with technical knowledge can build the cluster themselves, but those without experience can use Elastic Cloud. This is not an issue.

Correlation alerts is a feature I did not get the opportunity to work on. I have only theoretical knowledge but not practical knowledge.

We can use agentless approaches with a script in addition to agent-based approaches. We are building both agentless and agent-based solutions. Both are good. Agent-based approaches for fetching data work well. Both are functioning well.

Discovery is a feature we are using, and it works well. Attack is a feature I did not get the opportunity to try.

Elastic Search is very user-friendly, and we can easily integrate it with third-party models and other AWS  S3  buckets. It is very user-friendly for integrating with other third-party tools.

My overall review rating for this solution is ten out of ten.

I have been using Elastic Search  for the last five years.

I have a couple of use cases. First, I use it for logging purposes and observability logging of our product. In Azure , Elastic Search  has good support. Whenever I deploy any application, it automatically detects the application and tags the elastic log with it. This provides proper logging and observability to our application. That is my main use case. Another use case is making AI agents. In AI agents, I use it for vector search. Vector search means whenever I am searching anything in Elastic Search, which is a database, I can perform vector search on whatever I store in the database. Vector search is similarity search. For example, if I ask what are the petrol prices today, it will try to find similar items such as petrol, diesel, or similar things. If I ask about petrol, it will not only search for petrol but can also search for diesel because they are both liquid forms. Elastic Search has this search capability. I take the similarity search and after that add some of my algorithms to create the AI agent using that.

In traditional search, I get some log file and have to manually find information in it. For example, with text search, I type some keyword and manually have to open it in Notepad++ or any other similar tool. With Elastic Search, it is much better. I can search based on date ranges. For example, if I want to check the last one hour of data, I give the time frame and my application data appears there. If I want to search history, such as what happened one week ago with this application, and some customer provided some issue saying that one week back they received this issue, I can search the logs from one week back and go through those logs. Elastic Search has more search criteria. With different search criteria I can search it. I can also search based on context, where if I select the search in that time frame, it will search just before and after some context for me. That is also available in Elastic Search.

Hybrid search can be used programmatically as well. In Elastic Search, there is one user interface where I can provide a lot of things. That is one part of search. Hybrid search means if I want to search programmatically, I can search and get some data from Elastic Search and use it in my application. For example, if I am developing one agent, I definitely have to write some code and search some data using my program in Elastic Search. In that way, hybrid search is very useful. I can directly connect with Elastic Search database where I store all the data and get the data and use it in my application, wherever I want to use it. For example, if I am developing the AI agent, that is fine. If I want to just apply similarity search, I can also use it in my application.

Observability  is one part when I am deploying my application. When I deploy my application on the server in Azure , observability comes into the picture. Whenever I deploy my application, I need the log. Logging means observability, how my application is going on, whether I am getting any issues or whether I am getting any exception in the backend. That comes into the observability bucket. That is one use case of observability. The second is whenever I am developing RAG or AI agent. Whenever I am working on RAG, hybrid search comes into the picture, vector search, hybrid search. For security purposes, whenever it is deployed on Azure, it automatically handles security. I have worked with the cloud only, so I cannot tell much about security on this.

Regarding how I use Elastic Search in generative AI, I mostly use it for observability and RAG. Whenever I am deploying or creating the AI agent, I use RAG. Vector similarity search has been very helpful for me. I have different search criteria based on KNN or cosine similarity that I can use to search on Elastic Search database. The second is observability, which is also very good because most people are using Elastic Search because it is easy to use. As I explained before, I can give criteria by providing a date and time, and I can also see the graphs as well. Whenever I deploy the application, I can see usability graphs. It also shows the flow of data. Flow of data means if much data or some more operations are performed in this time frame, that graph will show as darker. I can easily see this because of small user interface presentations that are very good. I find it very useful in observability, log observability, and RAG development and AI agent development.

Hybrid search will be valuable.

Elastic Search is easy to use in Azure cloud. Mostly, my full company uses Azure cloud, so it is easy to use. Cost-wise, my company found Elastic Search is good. Cost matters. Based on cost and use cases, I found Elastic Search is good. Even compared to Splunk, Elastic Search has good easy-to-use user interface. Even non-technical people can easily search and easily observe the logs and easily track the applications. With Splunk, I found I have to be a little more technical in that area. There are key-based searches and some criteria that I have to remember. I found that difference between Splunk and Elastic Search.

Support-wise, it is good because I did not get much support work. Mostly my DevOps team handles it, but one or two times I did get support. There is a ticket creation option. Within the available time zone, somebody will be there to support me. Within two to three hours, somebody can help and try to resolve the issue.

Elastic Search is not specifically being used for certain purposes. I deploy Elastic Search database on the cloud and use cloud services so that nobody can attack. However, I do not use Elastic Search to resolve attack issues.

The basic main purpose of Elastic Search, as of now, I feel it can do more in the AI area. Sometime I saw that when I am developing RAG and have to generate the embeddings, which I call metadata, sometimes it tries to fail. That durability or issue handling should be improved, but apart from that, I did not find anything as of now. As per my use case, whatever I am using seems pretty good. Apart from that, some definitely improvement will be there. One improvement is that it should be faster. Whenever I am searching any logs, it takes much time. For example, if I open my log in Notepad or a similar tool, I can search the text within a second. With Elastic Search, it takes a little bit of time, ten to fifteen seconds. That can be improved. Sometimes, engineers take time to assign when I create a ticket.

Till now, I did not face any issue with the stability and availability of Elastic Search. It is not that the server is down. I faced issues such as some slowness. Whenever heavyweight logging will be there or heavyweight operations are performed, at that time, it will be a little slow. That sometimes also depends on cloud connectivity. Sometimes the cloud is only down, so it is very hard to perform my application better. I did not face any issue related to availability and other things. It is pretty good till now. The slowness is the one part, otherwise it is good.

Definitely, because I have very big applications in my company. It auto-scales up. Whenever I am deploying multiple instances of my application on a server, as I told, no need to give any configurations. For example, if I have five instances of my application I am deploying, automatically it will configure the five Elastic Search logs. Automatically it will create five Elastic Search configurations. Every application will have their own Elastic Search log. Auto-scaling wise, it is pretty good.

Support-wise, it is good because I did not get much support work. Mostly my DevOps team handles it, but one or two times I did get support. There is a ticket creation option. Within the available time zone, somebody will be there to support me. Within two to three hours, somebody can help and try to resolve the issue.

Sometimes, engineers take time to assign when I create a ticket.

I used Splunk. I have Splunk. Kibana, I think, merged with Elastic Search. I used Splunk and Kibana before. I am using pure Elastic Search now. For the last four to five years, I have been using pure Elastic Search. Before that, I was using Kibana and Splunk.

I am not aware of licensing and cost because I am not from the DevOps team. From a usability point of view, it is very easy to use and easy to plug with my application. I do not need extra configuration. Whenever I deploy my application on the server, I have to give the path of any observability tool such as Splunk or Kibana. Initially, I have to provide some extra configuration so that my log will appear on Elastic Search or Splunk. But nowadays, whenever I deploy my application, whatever logging I am doing is it will automatically connect with Elastic Search because Elastic Search has the capability to track. Whatever logging I am doing, whether it is SLF logging in Java, or in Python, whatever logging I am doing, basic logging is easily tracked by Elastic Search. No extra configuration is needed. It is just easy to plugin. I just deploy my application, and that is it. Automatically Elastic Search will track my log. No extra configuration is needed. I just have to make sure that I have Elastic Search services in my cloud and it should be enabled. That is all. Otherwise, it is easy to plugin.

Elastic Search is easy to use in Azure cloud. Mostly, my full company uses Azure cloud, so it is easy to use. Cost-wise, my company found Elastic Search is good. Cost matters. Based on cost and use cases, I found Elastic Search is good.

Elastic Search is easy to use in Azure cloud. Mostly, my full company uses Azure cloud, so it is easy to use. Cost-wise, my company found Elastic Search is good. Cost matters. Based on cost and use cases, I found Elastic Search is good. Even compared to Splunk, Elastic Search has a good easy-to-use user interface. Even non-technical people can easily search and easily observe the logs and easily track the applications. With Splunk, I found I have to be a little more technical in that area. There are key-based searches and some criteria that I have to remember. I found that difference between Splunk and Elastic Search.

Stack discovery is something I did not use till now. Whenever I am deploying my application on the cloud, and any attacks happen, I have some monitoring services in the cloud. Whenever something happens, if any attack happens to my Elastic Search database, it can happen through log injection. Something attackers can do a direct attack on my Elastic Search database and change some logs. This kind of scenario can come into the picture. I have some monitoring services deployed on the cloud. Whenever outside my company, outside of my company IP is trying to access my database or my data, that time automatically that monitoring alerts will be triggered and it will go to whoever is tagged into the mail. It will go to my higher manager and that mail will go to them. Regarding generative AI and how it will protect, nowadays, what is happening is that if I want to monitor this kind of attack, for that also, cloud is providing GenAI solutions. If this kind of attack comes, how automatically this GenAI resolves my problem, or how it suggests me to resolve the problem. That kind of solution I have already deployed on cloud.

I did not see much or connect with the support people much, but based on my experience, I would rate customer service as a four out of ten.

My overall rating for Elastic Search is eight out of ten.

I am a customer, and I use Elastic Search  to enhance our search capabilities in our applications.

Elastic Search  has excellent features, particularly its scalability and speed. What I appreciate most about Elastic Search is the ability to handle complex queries efficiently. I assess the relevancy of the search results by comparing it to hybrid search methods, such as vector and text searches, which helps ensure the accuracy of the results.

I see that there are areas in Elastic Search that have room for improvement, such as user documentation and onboarding processes.

Regarding the stability of Elastic Search, I find it to be quite robust, and I rate it a 9.

Regarding technical support, I would rate it an 8 because they are responsive and helpful.

The deployment took about two weeks, as we needed to ensure everything was configured correctly.

I compare Elastic Search with other solutions, such as OpenSearch  or Algolia , in terms of features and performance, which are quite impressive.

Elastic Search requires regular maintenance, including updates and patching to keep it running smoothly, and upgrades are straightforward to implement.

I have used Elastic Stream  for log investigation, which has been very helpful in diagnosing issues. We have about 50 active users in our organization.

As a developer, I use Elastic Search  in developing one of my applications, basically integrating the back-end with Elastic Search .

Our main use case for Elastic Search is for Logstash , which is a subset of Elastic Search that allows us to store logs and enables searching between logs with specific keywords in specific time ranges. Apart from that, we have our data stored in an index, and since Elastic Search is a NoSQL database, that's how we store the files in our databases.

The main objective of integrating Elastic Search is to transition from normal SQL databases to have faster searches and dynamic queries built around it, which makes the search much quicker. Since not all data is structured, we also need to handle unstructured data, and that's how Elastic Search has replaced our previous system.

The positive impact I've seen from using Elastic Search includes replacing conventional databases and being able to store much more unstructured data. In the future, if we need to include data not present in earlier systems, we can implement semantic or flyway changes with Elastic Search in place, allowing us to store unstructured data as is.

The most valuable feature of Elastic Search that I appreciate is the dynamic query building and the speed of result fetching, especially since we have an open-source version called OpenSearch  that we use in specific places due to the cost of storing data with Elastic Search.

Dynamic query building and result fetching are valuable because there are specific use cases where we need to build queries based on environment variables rather than having a generic query. This dynamic building helps address various business scenarios, especially considering customer product types and flags that may need inclusion or exclusion in the query. It allows me to create one query to accommodate multiple business cases and ensures that user-specific scenarios are included, with results already fetched for each.

Elastic Search has many features, including Kibana and Logstash , which we regularly use. However, one downside in our product is cost, as it can be expensive when maintaining multiple shards and indexes. Failures of shards or nodes can occur, and I can mention that cost and the upscaling of nodes or shards are areas needing improvement.

We haven't explored the hybrid search feature of Elastic Search, which combines vector and text searches, yet.

Scalability of Elastic Search presents disadvantages, particularly when handling minimal or production-level data. It manages high volumes of unstructured data well, but during performance tests involving one million requests at once, we encountered issues with shards and nodes not upscaling as needed, leading to crashes and minimal data loss, which isn't typical in real-world scenarios.

I have been working with Elastic Search for about 1.5 years.

Elastic Search is quite reliable for us, and despite identifying some very minute limitations, we still rely on Elastic Search.

Scalability of Elastic Search presents disadvantages, particularly when handling minimal or production-level data. It manages high volumes of unstructured data well, but during performance tests involving one million requests at once, we encountered issues with shards and nodes not upscaling as needed, leading to crashes and minimal data loss, which isn't typical in real-world scenarios.

I have not communicated with the technical support of Elastic Search at all up to this point.

Before Elastic Search, we used Couchbase, which is also a NoSQL database. Initially, it was free software integrated into our applications, but with its commercialization, we explored alternatives and found that Elastic Search would be a better fit than Couchbase.

We conducted some preliminary research to find a potential replacement for Couchbase while searching for NoSQL databases. The good documentation for Elastic Search on various websites helped us conclude that it would be an ideal fit. Although we considered the open-source version known as OpenSearch , we decided to integrate Elastic Search to explore its features, eventually determining it had much more powerful features, such as the Kibana dashboard and Logstash.

With respect to performance, we have seen a return on investment from Elastic Search. For example, the API response time has improved significantly, cutting the time down from about one or two minutes to around 50% faster, benefiting our downstream applications.