Splunk Observability Cloud

I have been using Splunk Observability Cloud  for the past one year in my career. Splunk Observability Cloud  has been introduced to our project for end-to-end monitoring for applications, providing complete visibility of applications, services, tech stacks, and CIs, which constitutes the whole monitoring solution for an entire application.

Previously, we were using different monitoring systems such as Dynatrace , the competitors of Splunk, and even Splunk Cloud Platform  or Enterprise platforms for logging alone. Now we have the entire solution under one name and one platform, which is Splunk Observability  Cloud, and that is why we mainly introduced Splunk Observability  Cloud to our project.

The UI is quite understandable, making it not as complex when compared to the other previous platforms I have worked on. Another thing I could specifically point out is that we can have entire visibility for the entire application performance when we look into Splunk Observability Cloud, and it is much easier to navigate across various aspects such as real user monitoring, application performance monitoring, or synthetic tests, making it stand above the other previous applications I have worked on previously.

One thing I should point out is that there are some auto-detectors which are defaultly present in Splunk. For example, if you are configuring a detector for AWS  RDS  service, you have an auto-detector which detects what the technology is, and you will have a readily available detector, needing only to configure your specific metrics on that, which is one advantage. The dashboards especially stand out, being different compared to the other platforms. Even previously, Splunk Enterprise also had dashboards, but this is different as we can have live metrics through the dashboards, which is quite impressive with how Splunk Observability Cloud has been introduced and it is performing better than the previous Splunk versions.

When we have too many detectors in place for one particular app, such as when I have created 50+ detectors through my account, the entire page becomes a bit loaded when creating the 51st detector, feeling heavy and taking time to load. Additionally, it throws random errors; for example, when we try to save one detector, it might throw some random error which is not even related, with something else being wrong, not that particular error, but the underlying root cause might be different. Sometimes the error is just "some problem occurred," and we are not able to point out what the real cause is.

This mainly happens when we have too many detectors or too many alerts in place rather than a standard number. One more thing is in the alert rules; if we have a main general alert, and instead of creating a new detector, we are adding a new rule under one detector, when the number of rules also increases, such as when we have 10 or 15 rules under one generic detector, that again creates the same kind of problem, taking some time to save that particular newly added rule, and it might not save at times, just keeps on spinning. Those are the two drawbacks which I spotted recently; other than that, everything looks perfect.

There was an outage which occurred about three or four months ago; that was the only outage I faced entirely in one year, and I believe that was a global outage from Splunk's side, which prevented us from logging in for a couple of hours. The Splunk team was working on it, and they resolved it within five or six hours, which was the only outage I faced in one year. Other than that, everything was smooth.

Splunk Observability Cloud is quite scalable compared to the other platforms I have worked on, and I do not find any difficulty in scaling up or even scaling down.

For particular kinds of issues which we were not able to resolve, we have raised Splunk tickets a couple of times before contacting the technical support or customer support. The support is an eight out of ten. The speed is actually quite good; they would respond within 48 hours, and the solutions they are giving are quite good, as we were able to solve most of the issues with their solutions.

I have used Dynatrace  and DataDog as alternatives to Splunk Observability Cloud.

The initial deployment of Splunk Observability Cloud is actually easy. With the clear documentation we have in place, it is quite straightforward. We even have examples of code snippets in the documentation, making it quite straightforward.

One or two people can manage the deployment; you do not need a team of five. I have myself worked on an entire project, and with one of my colleagues, I have worked on a much bigger project. I believe one or two people can easily manage the deployment process.

Splunk Observability Cloud has helped me reduce my mean time to detect. We have worked on around 80 applications last year for one particular client, and since the MTTR has improved drastically, they have given us 245 applications, which is around 150 applications added to the previous number of applications. This is definitely a performance improvement.

The pricing area I am not particularly aware of because that is centrally managed by the company which I work for. However, I feel that we are not spending too much on the licensing cost; it is manageable for how much we are working with currently for the number of applications we have at present, which is what I believe is not too much.

I would prefer Splunk Observability Cloud any day when comparing these solutions to Dynatrace and DataDog because the first thing is that the documentation Splunk has is perfect, and anybody who is new, even new to the platform, can gain knowledge reading through the documents, which are perfectly explained for configuring various kinds of technologies and integrating various kinds of technologies with Splunk Observability Cloud. The second thing is the UI, which is much more user-friendly compared to Dynatrace and DataDog.

The No-Sample Tracing helps me eliminate any blind spots in my data collection because we have particularly many services, and for example, using Mule, it is an added advantage to use no-sampling traces provided by Splunk Observability Cloud, giving us the exact points where the service is emitted and the exact spans between the two endpoints. It helps us break down where the actual issue is rather than just getting sample trace data and looking into each point entirely, which takes much time, providing a particular breakdown of that span and how it navigates across endpoints and pointing out the particular error which occurs when we access a service or when a service travels across two endpoints.

As of now, we are not using any AI tools in Splunk Observability Cloud, but we are planning to onboard them, considering that the number of applications we have is increasing day by day. We are planning to automate a few applications to generate the detectors and synthetics automatically as soon as we have the metrics in place. For that, we are developing code that can integrate with Splunk Observability Cloud platform and generate the results, which will be a time saver for us.

The recent UI changes have been more streamlined. Initially, the UI was a bit different, but later, I believe a couple of months ago, maybe one month before, they re-modified the menu options, and that has also resulted well for us in navigating across the panes.

I have utilized the ability to enrich data with custom metrics in Splunk Observability Cloud; I did it for custom metrics for AWS  services, and for a couple of MQ , IBM-based MQs, we have worked on custom metrics, integrating easily in both cases. My company may have partnerships with Splunk, but I am not sure of that. I would rate this review as a nine out of ten.

I am using Splunk Observability Cloud  as a log-based monitoring tool for my databases. We have ingested our database logs and OS system logs into Splunk Observability Cloud  and are creating dashboards and alerting features over those alerts. One of my major use cases is that all kinds of databases I am currently working with have database logs that capture all information, warnings, and error messages. These database logs are moving to Splunk Observability  Cloud. The first use case is that I no longer need to maintain a long list of flat files on my server for all those logs. Those can be directly ingested into Splunk Observability  Cloud. The benefit I am seeing from here is that I can get pattern-based analysis of what kind of errors I am commonly getting and what the date patterns of those errors are. I can get dashboards over that and I can also create alerts. I can also incorporate those alerts with some back-end Git  workflow for automatic remediation. This is one of the solutions.

Another use case for Splunk Observability Cloud that we are seeing is that there are multiple times when there is a requirement to publish some kind of data. So instead of publishing an alert if those data breaches occur or if some kind of dashboard needs to be created, instead of sending data directly to the users, if that data is not PII, we are also ingesting that into Splunk Observability Cloud in a JSON format and then again, dashboards and other alerting can be created. These two are the main major use cases for which I am using Splunk Observability Cloud.

With the help of the alerting and observability mechanism, resiliency, and automatic automation of issue remediation based on alerts and workflows, it actually reduces the cost and increases the uptime of my system and customer satisfaction. There are multiple indirect benefits I am getting when using Splunk Observability Cloud.

Currently, with the growth of the organization, I am seeing an increasing use of Splunk Observability Cloud in a more dynamic way. We are continuously creating new dashboards, ingesting logs in JSON, and trying to bring the best value out of it. I am seeing a dynamic and drastic increase in the use of Splunk logs and the Splunk data we are ingesting.

There are two aspects to expanding the usage. Organic growth of the environment actually puts new systems into Splunk Observability Cloud, and exploring new opportunities for what all can also be ingested into Splunk Observability Cloud. Previously, I can see that memory dumps are there. We are also looking at whether we can ingest memory dumps so that if the system is about to crash, those memory dumps can be captured into Splunk Observability Cloud so that it can create alerts over that and I can also perform analysis. I can also see if any other system is facing the same kind of memory dump issues. So that maybe it is one alert for one system for me, but for the complete farm, there may be different servers with different teams or business units facing the same issues. When I have Splunk Observability Cloud on all systems, I can actually create a consolidated report and see that this is the pattern which particular farms are having this kind of issues, and maybe something is broken. This is the way the plan is to increase the availability or the usage of Splunk Observability Cloud.

The performance and speed are valuable. Previously when Splunk offered the enterprise solution, I needed to install Splunk and maintain my local server. There was a limitation that only a certain number of servers could be supported in one instance and I would need to have multiple instances if I was in an enterprise system setup. When I am in the cloud, a single instance can support N number of systems. It is pretty fast, no matter how much data is there. Dashboards are pretty good with multiple functions available. The alignment or integration that can trigger automatic solutions with the workflow for automatic remediation of the alerts is the best thing. These three or four things are the best Splunk Observability Cloud features that I am seeing.

The point in time alerting, the point in time data capture, and automatic remediation with the integration of good workflows or Ansible  workflows is definitely the key to any resiliency and increasing the uptime of any system.

After moving to Splunk Observability Cloud, it is almost zero downtime. We never face downtime because when I was in the enterprise setup, I needed to maintain my servers and maintain hygiene of vulnerabilities, patches, and all. Now when I am in the cloud, everything is automatic. Almost zero downtime plus the perfect alerting feature and log-based analysis are available. Metrics alerting is also there in Splunk Observability Cloud through queries. This is one of the features that keeps me updated with the current health of my system and helps me to keep my system up and running fine and available for my customers.

Splunk Observability Cloud incorporated a new AI agent feature that is really good. Sometimes I need to create queries and Splunk queries for filtering the data and some pattern-based analysis. This agent is really good in helping me and suggesting the queries. This means I do not need to have a Splunk expert or Splunk query expert. I can just ask that agent that I need pattern-based analysis or I need to create this kind of filters for this kind of data and it can suggest to me. Once it suggests a sample query to me, I can do the tweaking and I can have my data ready. It actually reduces my time to perform my analysis and to reach the conclusion about what exactly is causing issues in my system and what are the repetitive issues in my system. This AI feature really helps for newcomers to Splunk Observability Cloud to perform deep diving analysis with the data captured by it.

Custom metrics are valuable. In Splunk Observability Cloud, some infra-level metrics are not available, but through custom metrics, I can achieve it. This is an add-on feature that Splunk Observability Cloud is providing and without any additional monitoring tool. If that feature was not there, then I would need to plan some other monitoring tool for metrics-based alerting, but this custom one helps me to achieve it in the same monitoring tool. The consolidation and integration of metrics-based alerting and log-based alerting in a single tool is actually the lovable feature. I do not need to worry about or look for multiple tools. I can have my own data and own health available in a single tool, in a single view.

The dashboards are good, but the only limitation I see currently is that they need particular formats only to create a dashboard. They need to have a particular JSON format or time series format. This sometimes creates additional work for me so that when I am ingesting logs in Splunk Observability Cloud, it should be in a specific format. Either Splunk Observability Cloud should have multiple formats available or multiple dashboards available for different kinds of formats. At least Splunk Observability Cloud has everything available at a Splunk level. They can do some kind of analysis and see what are the major top ten or top twenty types of logs they are getting and they can have dashboards according to those logs. Instead of forcing customers to design their logs in the way of Splunk Observability Cloud, Splunk Observability Cloud can create dashboards based on the customer requirement. This will actually ease things up for the end users.

The current dashboards are good. The feedback is that Splunk Observability Cloud is forcing me to modify my logs that I am ingesting in Splunk Observability Cloud in a specific format. If Splunk Observability Cloud can leverage it and make it open for any format, that would be great. If that is not feasible, at least the top ten or top twenty logs that Splunk Observability Cloud is getting should be readable by Splunk Observability Cloud without any changes. That actually is one of the major feedback items I can provide which can actually ease the life of the end users or any layman. As a newcomer to Splunk Observability Cloud, I may not know JSON. I now need to hire someone or I need to look for someone who knows JSON and who can convert my logs into JSON format and then I will ingest them into the logs if I want to create a dashboard. If I do not want to create a dashboard, that is okay. On the other hand, Splunk Observability Cloud is giving me a usability and easy to go interface, but for a dashboard, I need to have an understanding of JSON so that I can ingest the log in JSON format. That is a dilemma that they have and they should work on.

Currently, Splunk Observability Cloud is not the only solution which any organization is using. There is also Grafana  and PagerDuty. If Splunk Observability Cloud can plan some kind of integration with PagerDuty and Grafana , then those things can be controlled from a single position and if something else is happening at one location, it can update things at all levels. That can also bring great value to the users. Currently, I have to maintain three systems separately, but if some kind of integrations can be developed with these three vendors, then that can be a great thing because all these three have now become the industry pillars or industry standards for observability and resiliency.

I have been working with it for the last two years. Before that, it was an enterprise solution. Now it is cloud-based.

I cannot relate any stability issues to my experience with Splunk Observability Cloud.

Scalability is pretty smooth. I just need to deploy the Splunk forwarder and the config file that specifies which servers it should connect to and it will get connected. My data will start populating. It is pretty straightforward. I do not see any challenges there, even when it was in enterprise and now when it is in the cloud. The deployment and onboarding of new servers and ingesting the logs is pretty straightforward. Anybody can learn it within a day without having any prior knowledge.

We have raised multiple questions when we face any issues. Our support is prompt and usually within a day, I will get my answers.

Previously I was on Splunk Enterprise. I have been using Splunk for seven to eight years before we moved to the cloud in the last eighteen months.

The initial setup is pretty smooth. I just need to deploy the Splunk forwarder and the config file that specifies which servers it should connect to and it will get connected. My data will start populating. It is pretty straightforward. I do not see any challenges there, even when it was in enterprise and now when it is in the cloud. The deployment and onboarding of new servers and ingesting the logs is pretty straightforward. Anybody can learn it within a day without having any prior knowledge.

I appreciate that your organization collects reviews about the product so that it can be shared with the vendor or the product owner as appreciation or as feedback for improvement. Everything has been smooth in my experience. I would rate this product a ten out of ten.

The main use case with Splunk Observability Cloud  is to capture the logs from the SD-WAN in order to check the health of the network and the flow of data from different sources to the central place.

The best feature of this product is the latency and processing of all the telemetry that is being received, which gives full visibility at the right time.

One cannot protect and operate what they don't know. When there is this observability, it helps to see exactly what is present, the problems that may exist, and hence, it increases digital resilience by having proactive actions ahead, which increases the availability of the service.

The teams have utilized the ability to enrich data with custom metrics, as this enrichment is one of the key aspects used to have a clear understanding of which assets are being attacked, enabling necessary actions to be taken. The data has been enriched by adding customized information from customers' databases from different sources.

The pricing would be one area for improvement.

I have used the SIEM  solution since 2019 and have had experience with Splunk Observability Cloud  for the last year.

I would rate their customer service and technical support an eight out of ten.

I work for SI, and we deliver to different organizations based on their requirements. We are responsible for implementation, so we implement and they see the value out of it.

Splunk Observability  Cloud has improved the operational performance of our clients.

It is expensive.

The AI component is one of their strengths; currently, most competitors are moving in the same direction. As SI professionals, we are seeing different improvements in the AI domain for different products, and they are at the leading edge with many vendors following them.

My overall rating for Splunk Observability Cloud would be a seven out of ten.