Elastic Cloud (Elasticsearch Service)

I have been using Elastic Search  for the last five years.

I have a couple of use cases. First, I use it for logging purposes and observability logging of our product. In Azure , Elastic Search  has good support. Whenever I deploy any application, it automatically detects the application and tags the elastic log with it. This provides proper logging and observability to our application. That is my main use case. Another use case is making AI agents. In AI agents, I use it for vector search. Vector search means whenever I am searching anything in Elastic Search, which is a database, I can perform vector search on whatever I store in the database. Vector search is similarity search. For example, if I ask what are the petrol prices today, it will try to find similar items such as petrol, diesel, or similar things. If I ask about petrol, it will not only search for petrol but can also search for diesel because they are both liquid forms. Elastic Search has this search capability. I take the similarity search and after that add some of my algorithms to create the AI agent using that.

In traditional search, I get some log file and have to manually find information in it. For example, with text search, I type some keyword and manually have to open it in Notepad++ or any other similar tool. With Elastic Search, it is much better. I can search based on date ranges. For example, if I want to check the last one hour of data, I give the time frame and my application data appears there. If I want to search history, such as what happened one week ago with this application, and some customer provided some issue saying that one week back they received this issue, I can search the logs from one week back and go through those logs. Elastic Search has more search criteria. With different search criteria I can search it. I can also search based on context, where if I select the search in that time frame, it will search just before and after some context for me. That is also available in Elastic Search.

Hybrid search can be used programmatically as well. In Elastic Search, there is one user interface where I can provide a lot of things. That is one part of search. Hybrid search means if I want to search programmatically, I can search and get some data from Elastic Search and use it in my application. For example, if I am developing one agent, I definitely have to write some code and search some data using my program in Elastic Search. In that way, hybrid search is very useful. I can directly connect with Elastic Search database where I store all the data and get the data and use it in my application, wherever I want to use it. For example, if I am developing the AI agent, that is fine. If I want to just apply similarity search, I can also use it in my application.

Observability  is one part when I am deploying my application. When I deploy my application on the server in Azure , observability comes into the picture. Whenever I deploy my application, I need the log. Logging means observability, how my application is going on, whether I am getting any issues or whether I am getting any exception in the backend. That comes into the observability bucket. That is one use case of observability. The second is whenever I am developing RAG or AI agent. Whenever I am working on RAG, hybrid search comes into the picture, vector search, hybrid search. For security purposes, whenever it is deployed on Azure, it automatically handles security. I have worked with the cloud only, so I cannot tell much about security on this.

Regarding how I use Elastic Search in generative AI, I mostly use it for observability and RAG. Whenever I am deploying or creating the AI agent, I use RAG. Vector similarity search has been very helpful for me. I have different search criteria based on KNN or cosine similarity that I can use to search on Elastic Search database. The second is observability, which is also very good because most people are using Elastic Search because it is easy to use. As I explained before, I can give criteria by providing a date and time, and I can also see the graphs as well. Whenever I deploy the application, I can see usability graphs. It also shows the flow of data. Flow of data means if much data or some more operations are performed in this time frame, that graph will show as darker. I can easily see this because of small user interface presentations that are very good. I find it very useful in observability, log observability, and RAG development and AI agent development.

Hybrid search will be valuable.

Elastic Search is easy to use in Azure cloud. Mostly, my full company uses Azure cloud, so it is easy to use. Cost-wise, my company found Elastic Search is good. Cost matters. Based on cost and use cases, I found Elastic Search is good. Even compared to Splunk, Elastic Search has good easy-to-use user interface. Even non-technical people can easily search and easily observe the logs and easily track the applications. With Splunk, I found I have to be a little more technical in that area. There are key-based searches and some criteria that I have to remember. I found that difference between Splunk and Elastic Search.

Support-wise, it is good because I did not get much support work. Mostly my DevOps team handles it, but one or two times I did get support. There is a ticket creation option. Within the available time zone, somebody will be there to support me. Within two to three hours, somebody can help and try to resolve the issue.

Elastic Search is not specifically being used for certain purposes. I deploy Elastic Search database on the cloud and use cloud services so that nobody can attack. However, I do not use Elastic Search to resolve attack issues.

The basic main purpose of Elastic Search, as of now, I feel it can do more in the AI area. Sometime I saw that when I am developing RAG and have to generate the embeddings, which I call metadata, sometimes it tries to fail. That durability or issue handling should be improved, but apart from that, I did not find anything as of now. As per my use case, whatever I am using seems pretty good. Apart from that, some definitely improvement will be there. One improvement is that it should be faster. Whenever I am searching any logs, it takes much time. For example, if I open my log in Notepad or a similar tool, I can search the text within a second. With Elastic Search, it takes a little bit of time, ten to fifteen seconds. That can be improved. Sometimes, engineers take time to assign when I create a ticket.

Till now, I did not face any issue with the stability and availability of Elastic Search. It is not that the server is down. I faced issues such as some slowness. Whenever heavyweight logging will be there or heavyweight operations are performed, at that time, it will be a little slow. That sometimes also depends on cloud connectivity. Sometimes the cloud is only down, so it is very hard to perform my application better. I did not face any issue related to availability and other things. It is pretty good till now. The slowness is the one part, otherwise it is good.

Definitely, because I have very big applications in my company. It auto-scales up. Whenever I am deploying multiple instances of my application on a server, as I told, no need to give any configurations. For example, if I have five instances of my application I am deploying, automatically it will configure the five Elastic Search logs. Automatically it will create five Elastic Search configurations. Every application will have their own Elastic Search log. Auto-scaling wise, it is pretty good.

Support-wise, it is good because I did not get much support work. Mostly my DevOps team handles it, but one or two times I did get support. There is a ticket creation option. Within the available time zone, somebody will be there to support me. Within two to three hours, somebody can help and try to resolve the issue.

Sometimes, engineers take time to assign when I create a ticket.

I used Splunk. I have Splunk. Kibana, I think, merged with Elastic Search. I used Splunk and Kibana before. I am using pure Elastic Search now. For the last four to five years, I have been using pure Elastic Search. Before that, I was using Kibana and Splunk.

I am not aware of licensing and cost because I am not from the DevOps team. From a usability point of view, it is very easy to use and easy to plug with my application. I do not need extra configuration. Whenever I deploy my application on the server, I have to give the path of any observability tool such as Splunk or Kibana. Initially, I have to provide some extra configuration so that my log will appear on Elastic Search or Splunk. But nowadays, whenever I deploy my application, whatever logging I am doing is it will automatically connect with Elastic Search because Elastic Search has the capability to track. Whatever logging I am doing, whether it is SLF logging in Java, or in Python, whatever logging I am doing, basic logging is easily tracked by Elastic Search. No extra configuration is needed. It is just easy to plugin. I just deploy my application, and that is it. Automatically Elastic Search will track my log. No extra configuration is needed. I just have to make sure that I have Elastic Search services in my cloud and it should be enabled. That is all. Otherwise, it is easy to plugin.

Elastic Search is easy to use in Azure cloud. Mostly, my full company uses Azure cloud, so it is easy to use. Cost-wise, my company found Elastic Search is good. Cost matters. Based on cost and use cases, I found Elastic Search is good.

Elastic Search is easy to use in Azure cloud. Mostly, my full company uses Azure cloud, so it is easy to use. Cost-wise, my company found Elastic Search is good. Cost matters. Based on cost and use cases, I found Elastic Search is good. Even compared to Splunk, Elastic Search has a good easy-to-use user interface. Even non-technical people can easily search and easily observe the logs and easily track the applications. With Splunk, I found I have to be a little more technical in that area. There are key-based searches and some criteria that I have to remember. I found that difference between Splunk and Elastic Search.

Stack discovery is something I did not use till now. Whenever I am deploying my application on the cloud, and any attacks happen, I have some monitoring services in the cloud. Whenever something happens, if any attack happens to my Elastic Search database, it can happen through log injection. Something attackers can do a direct attack on my Elastic Search database and change some logs. This kind of scenario can come into the picture. I have some monitoring services deployed on the cloud. Whenever outside my company, outside of my company IP is trying to access my database or my data, that time automatically that monitoring alerts will be triggered and it will go to whoever is tagged into the mail. It will go to my higher manager and that mail will go to them. Regarding generative AI and how it will protect, nowadays, what is happening is that if I want to monitor this kind of attack, for that also, cloud is providing GenAI solutions. If this kind of attack comes, how automatically this GenAI resolves my problem, or how it suggests me to resolve the problem. That kind of solution I have already deployed on cloud.

I did not see much or connect with the support people much, but based on my experience, I would rate customer service as a four out of ten.

My overall rating for Elastic Search is eight out of ten.

I am a customer, and I use Elastic Search  to enhance our search capabilities in our applications.

Elastic Search  has excellent features, particularly its scalability and speed. What I appreciate most about Elastic Search is the ability to handle complex queries efficiently. I assess the relevancy of the search results by comparing it to hybrid search methods, such as vector and text searches, which helps ensure the accuracy of the results.

I see that there are areas in Elastic Search that have room for improvement, such as user documentation and onboarding processes.

Regarding the stability of Elastic Search, I find it to be quite robust, and I rate it a 9.

Regarding technical support, I would rate it an 8 because they are responsive and helpful.

The deployment took about two weeks, as we needed to ensure everything was configured correctly.

I compare Elastic Search with other solutions, such as OpenSearch  or Algolia , in terms of features and performance, which are quite impressive.

Elastic Search requires regular maintenance, including updates and patching to keep it running smoothly, and upgrades are straightforward to implement.

I have used Elastic Stream  for log investigation, which has been very helpful in diagnosing issues. We have about 50 active users in our organization.