Elastic Cloud (Elasticsearch Service)

I developed a chatbot with text summarization and question answering capabilities. I need to summarize multiple PDFs, and I have a database in Google Cloud Storage  where I perform keyword matching with Elastic Search  using exact keyword matching.

I have different clients, but I use Elastic Cloud (Elasticsearch Service)  for one specific client. For that one job, Elastic Cloud (Elasticsearch Service)  is the main tool because I am using an Elastic Search  strategy instead of a vector database.

Scalability is valuable to me. I have 50,000 PDF JSON files that contain my metadata, and I am really glad to use Elastic Cloud (Elasticsearch Service) for this volume without any issues. From a startup's perspective, I would say that until 10 GB of storage, there is no problem whatsoever.

Application-wise, everything was easy to work with. I really appreciated their dashboard because everything was clear, and it was easy to implement.

Because I am pursuing a PhD and work under the university, my university has an agreement with AWS , which makes it essentially free and easier to use. In the AWS  ecosystem, everything is connected and I can control everything without uncertainty about what is happening behind the scenes. However, when using Elastic Cloud (Elasticsearch Service), I connected it to Google Cloud  but I am paying separate receipts. Over the last two months in October and November, I paid two separate invoices that are not connected to Google Cloud , which I did not appreciate.

Google Cloud has a nice interface that gives me full control of pricing and billing. I can see daily, weekly, and monthly breakdowns with bar charts, and I can track exactly how much I spent during any period. Elastic Cloud (Elasticsearch Service) does not have such a tool for billing visibility. Since I am handling significant amounts of money and am responsible for this task within my company, I have high expectations for pricing and billing transparency. I would appreciate the ability to set a spending limit, such as uploading 200 euros, and receive notifications when reaching 50% of that limit. These notifications could appear on the dashboard, in the application, or via email. It would be valuable to see a timeline of my spending.

I would characterize the pricing as somewhat expensive. I did not use competitors extensively, so I may have a bias about this. The pricing of large language models is not expensive—I use Anthropic's Claude or Google's Gemini, which are state-of-the-art models. However, I am uncertain whether I have a bias about Elastic Cloud (Elasticsearch Service) pricing. It is not extraordinarily expensive, but when I compare it with the cost of using large language models or Google Cloud storage, it is quite expensive.

A couple of days ago, the Elastic team reached out to me. We have been regularly using the service since April, and 10 days ago at the beginning of December, I deleted my hosted deployments because I did not like the idea of paying when I am not actively using Elastic Cloud (Elasticsearch Service). They informed me that there is a serverless option available. Before Christmas, I want to try it to see how it works, as I am uncertain about the serverless concept and whether it will provide the same functionality that I use with the hosted deployment.

I have been using this since April.

I have experienced no issues whatsoever in the last five or six months. Whenever I perform my searches, and because my application is active with clients using it, there has been no feedback about problems. During my tests, I did not observe any lag or delay.

I would give them a rating of ten. They were extremely helpful, kind, and communicative. Three people from the Elastic team spoke with me, and they were genuinely trying to solve the problem and understand my expectations. It was really excellent, and I recommend them highly.

I used the hosted deployment version, not the serverless option yet. The hosted deployment took me 10 to 15 minutes to set up. It was very easy and primarily involved indexing. I generated Python code on my local computer within minutes and pushed everything with the indexing. It took about 15 to 20 minutes total, though this is related to the size of my folder. I was indexing around 10,000 PDFs and creating metadata JSON files. The process was easy and fast.

I also use AWS daily in their ecosystem, which contains everything I need. I use Google Cloud primarily because of the pricing, as I must consider profit margins.

My team is small, consisting of about four or five people. On Elastic Cloud (Elasticsearch Service), only two of us work with it, but I am the one who uses it daily.

So far, I have not performed any maintenance.

Elastic Cloud (Elasticsearch Service) focuses on exact keyword matching. This means they do not address semantic similarity well. For example, if I use a word and then use another word with the same meaning in a sentence but not a synonym or similar word, Elastic cannot understand this semantic similarity, which is important for my chatbots. This is why I was using vector databases, as they focus on semantic similarity of words and tokens, whereas Elastic looks for exact word representation.

The team mentioned that hybrid search is an option. I have my own vector database that I use daily as a personal solution, and I could give hybrid search a chance, but I have not tried it yet.

I would rate my overall experience as seven out of ten. Two points are deducted for the pricing, which was higher than my expectations. The pricing has been significant over the last two months and was considerably more than I anticipated. My overall review rating for this service is nine out of ten.