I received information from your team regarding a peer review of Honeycomb Enterprise. As an observability engineer using Honeycomb Enterprise extensively, I can provide substantial input. My primary tasks involve OpenTelemetry, where I send numerous traces and metrics to Honeycomb Enterprise and use it for visualizations of how our code is functioning and how data moves through our systems.

We have identified many problems using Honeycomb Enterprise, such as delays in the system where latency is observed by Honeycomb Enterprise traces, which we then investigate further and resolve. We have our SLA and SLOs configured on Honeycomb Enterprise, allowing us to monitor our reliability score. Additionally, we send many metrics to the platform. In my previous organization, we used it only for traces, but in my current organization, I use it extensively for traces, metrics, SLA, and SLO work. We also use Signoz for infrastructure-related APM metrics and other things, but we cannot use it for Honeycomb Enterprise capabilities.

Honeycomb Enterprise has genuinely helped us. We are not using Honeybee command line, which is a native solution by Honeycomb Enterprise. Instead, we use OpenTelemetry so that we can switch vendors easily. Our complete codebase now has proper telemetry data sending into Honeycomb Enterprise.

We have provided traces for each of the functions we are using. If we need more in-depth insights into what happens inside the code, we set up additional telemetry data points in the code. When you set those telemetry data points there in the code, you send those traces to Honeycomb Enterprise and then we can examine them.

In terms of customer usage, observing how customers are using our solution is very helpful in debugging aspects and also helpful in cost prioritization aspects. If a customer is not using our solution extensively, we can scale it down a bit in terms of servers where our solution is running. This has given us a better approach, particularly for Lambdas where Lambdas are deployed, and we can save considerable costs.

If traffic is not very high and we are able to see metrics where traces are not falling a lot, we can scale down our instances and save our AWS billing. If there is a sudden spike, we can get that monitored via the SLI and SLOs that I have configured. If there are sudden errors, I can scale the instance up.

If there is a genuine bug in the code, Honeycomb Enterprise is very useful for debugging the issues. Even a data-driven issue can be fixed using it because we get all the data when a user is providing any request. We actually get the input data in Honeycomb Enterprise and that way we can solve it very fast by seeing what query they hit or what workflow they have done.

Tracing is undoubtedly one of the best features they have developed. Tracing is basically the story in the code showing how the data is moving through. That is something which is extremely good. I can vouch for Honeycomb Enterprise, and that is the reason I brought Honeycomb Enterprise as our solution in my current organization.

That is the best thing Honeycomb Enterprise has done. In the past, in my previous organization, we were using something called GCID where if you send data, GCID is basically each document we are receiving. We were using an enterprise solution there called Smarsh. In that company, we were getting almost five petabytes of data per day, so that is the volume I was dealing with. Honeycomb Enterprise was actually really great in terms of tracing. We had a sampling rate and we were not sending all the traces, but the best part is we were able to add more cardinal events, such as GCID, which was getting mapped in each and every document. We were sending those metrics blindly and we did not have to think more about it. The amount of engineering that Honeycomb Enterprise has done is actually handling it very well. At that scale also, I can vouch for Honeycomb Enterprise that it manages traces beautifully.

That money is well spent. We do not have any complaints because tracing is something which is extremely good. How they cater to it and how they manage it is excellent. It is an extremely cost-effective solution right now in the market as compared to its peers like DataDog and other companies.

Throughput is something which I was using previously. There were around twenty million ingestions in my previous organization, obviously. Here the scale is not very high where I can challenge the engine. In my previous organization, when there were twenty plus million ingestions of data every day, in those kinds of scenarios, Honeycomb Enterprise actually stood very well. The major solutioning we were using was for traces. We were not using it for metrics. So if there are more data points, there might be a crash in Honeycomb Enterprise. However, honestly, I have not used it at that scale, so I cannot comment exactly on that point. I have used it for metrics and that performed well, but not on a very high volume of data where I can challenge the engine itself on how it is processing.

I would say it worked pretty well for mid-tier or similar tier companies. I do not have any complaints. But for enterprise solutioning, probably when a lot of data points come in, DataDog obviously has an edge there. They are actually the market leaders around it. They have better metrics, APM, and other aspects there.

Honeycomb Enterprise's tracing solution stands out. Whatever they have built, they have built an excellent product there. Honeycomb Enterprise actually works very well with traces. When you send traces, you will get the complete view of the life of the code and how it has been executed.

The only complaint I have is that even though we are on a paid tier where we are paying one hundred thirty dollars per month, we are still lacking the amount of ingestion we have to do. It counts each and every event. We cannot blindly send the performance metrics and other things directly to Honeycomb Enterprise as it will raise our bills. They have some buffer protection, but that only happens twice or three times a month. After that, the number of events will get billed accordingly.

On the negative side, dashboarding is not that great. I have used solutions like Signoz which are open-source and Grafana. In those platforms, where we have open-source solutions where we are sending in the metrics, we are getting a complete dashboard and a complete solution around it where we can analyze everything in one shot. That is not something which is really great with Honeycomb Enterprise.

These kinds of solutions in Honeycomb Enterprise feel very difficult to build because I think it is an over-engineered product in terms of metrics. For traces, I have to give it to them. They have done an excellent job.

The only thing is the UI is not that great for Honeycomb Enterprise. That is something where it is not very appealing when I am looking into it. It is more of an engineered thing. I am an engineer and I understand this. You cannot make the UI very beautiful. However, if you use its peers, such as DataDog or Signoz, there are very interactive dashboards and other things. The dashboard is not that great, honestly, in Honeycomb Enterprise.

There is a cold start problem because we were using something called Lambda in AWS. The cold start issues basically occurred because we had not provided proper RAM to our Lambda function. Because of that, the Boto3 boot up was taking a lot of time. That was caught during these tracing sessions when we added the traces. We were able to identify that the Boto3 startup, the init part of Boto3, was taking more than twelve seconds or ten seconds to boot up. That is basically a very huge latency when it comes to a SaaS product.

Those are the major things. One thing they should do is discontinue Honeybee CLI. Right now, no one is using Honeycomb Enterprise like Honeybee CLI as per my understanding. Very few people are using it, and only POC kind of things are happening on this. People are shifting to OpenTelemetry. They should actually discontinue that product. It does not make much sense because they are trying to bring a dependency on people who are using that and they want to lock them into Honeycomb Enterprise because once the code is telemetered, it is very difficult to change it again.

However, now with cloud and everything we have, it is very easy to make those changes. There is no mode there. If there are engineers around it who are managing those things, they can actually shift to a better place where they can provide more value around it. Because people will eventually move to OpenTelemetry where they will set up their OpenTelemetry system, one OTel collector, and then OTel collector will send everything to Honeycomb Enterprise. People are moving out of these enterprise solutions mostly and they are using OpenTelemetry day in and day out. Most of the architects I talk to in terms of these things always talk about this.

Personally, I have also not used Honeycomb Enterprise CLI. I have read a lot of things and articles around it. They are actually maintaining it very well it seems, but I think they should discontinue it. That is just my opinion based on my observations. I may be wrong, I am not sure. This is what I have observed during communication channels, followed by the usage I have done, and followed by people I talk around. All have similar opinions.

I have been using this for almost three and a half years now.

I have not had any stability issues.

The setup is pretty easy. That is straightforward. I do not have any complaint there. The documentation is well written. I set up Honeycomb Enterprise traces just for a POC kind of thing within less than thirty to forty minutes just for a small code. The actual production code took one day, but that is because the code was long. That was easy. It was not very difficult. Things were very clear around it regarding how to generate the API keys to send those things and how to send it.

The load balancer API of Honeycomb Enterprise is extremely good. It accepts everything. I do not have any concerns there. I have never brought it down. I am not sure about the reliability aspect of it, but it accepts everything and I have never seen it down in my experience. I do not have any concerns there.

What we have done in terms of ROI using Honeycomb Enterprise, I still remember one use case where we reduced our overall latency because there was one Boto3 library which was not getting the expected performance.

Honeycomb Enterprise played a vital role in identifying the problems in the initial calls itself. That has actually saved us a lot of incidents. Previously, there were a lot of customer complaints coming in. I have taken one step at a time and I have added all these things in our code and followed by the customer complaints are almost ninety to ninety-two to ninety-three percent down as per the data I have.

Honeycomb Enterprise has actually helped us in various ways. The major complaints were around latency. We were able to switch our databases also because there was one problem that was identified by Honeycomb Enterprise where we were trying to fetch more than one megabyte of data from DynamoDB at one point in time. However, the problem was there is a hard limit set to DynamoDB to fetch these kinds of data.

That was a serious problem because customers were expecting data what they want to see on their screen. These kinds of issues were identified during our initial checks on DynamoDB and we were able to move to Snowflake. This has given us better solutioning around it.

In those scenarios where you are not getting the complete data to the customer, it will cap the data to one megabyte.

For tracing solution, definitely, I will always suggest Honeycomb Enterprise is a better solution. Other observability solutions out there are expensive and they are not that reliable in terms of tracing, to be honest.

If there is a genuine bug in the code, then Honeycomb Enterprise is also very useful for debugging the issues. Even a data-driven issue can be fixed using it because we get all the data when a user is providing any request. We actually get the input data in Honeycomb Enterprise and that way we can solve it very fast. We can see what kind of query they hit or what kind of workflow they have done.

I think people are very resistant to adopting Honeycomb Enterprise. If you see the community, I am in the Honeycomb Enterprise community right now. I usually ask a lot of questions there. Now that AI came, the questions volume has decreased significantly. However, previously it was a very healthy community where a lot of questions came in. If you go through those questions, the major things are around how to set up things. It is not always about the metrics cardinality or how things are not working around it or if there are any solutions.

I would say it worked pretty well for mid-tier or similar tier companies. I do not have any complaints. For enterprise solutioning, probably when a lot of data points come in, DataDog obviously has an edge there. They are actually the market leaders around it. They have better metrics, APM, and other aspects there.

My overall review rating for Honeycomb Enterprise is eight out of ten.

I am part of the performance engineering practice, and I lead the performance engineering practice at my current employer. We use Honeycomb Enterprise for tracing, which is application performance management in short. Our client has several APM tools such as Datadog, and in addition to Datadog, we only have the monitoring capacity of the counters. We do not have the agent-level monitoring which Honeycomb Enterprise is providing, where we can see the traces for each call being made by the software to trace where it is spending the time. For that gap, they have Honeycomb Enterprise in addition to Datadog. We use Honeycomb Enterprise for the same purpose, as Honeycomb hooks into our applications and tells us the traces where the request is spending the time.

We have Datadog here and often we get restrictions related to cardinality on Datadog because of their billing systems. They have limitations of cardinality, and that is the impact. That is how we can compare the impact.

Another thing I want to add here is that the team here had tried to use Honeycomb Enterprise earlier for tracing, but they faced issues. They could not get proper tracing with Honeycomb Enterprise at that time. That is what I have been given as feedback.

My main focus is on the tracing part. We have a microservices architecture with multiple microservices, and we want to see how when the request flows across multiple microservices, where the time gets spent.

We mostly look at the time the requests spend. Honeycomb Enterprise would capture a trace and span, and from there, we look at the time, the milliseconds or seconds that get spent at a particular request. That is what we look at and what we are interested in.

I did not get a chance to utilize the BubbleUp feature, and I think this is the most famous feature for SREs. We have not been able to utilize BubbleUp.

Dynatrace has this PurePath technology where they have the ability to give end-to-end tracing, which is a very robust feature. AppDynamics also has certain good features such as we could deploy that on our database systems and it would give an entire picture of our database, showing which application is creating how much load. From a pros perspective, Honeycomb Enterprise could be a better candidate with high cardinality; when there are too many unique values, Honeycomb Enterprise could be more beneficial there.

I have used better tools, I would say. I would not say that I prefer Honeycomb Enterprise as much. I have used Dynatrace, and I found it more comprehensive, and AppDynamics and other tools. These tools can also provide good information, but I find other tools better.

Most of the products, I would say, such as Dynatrace or AppDynamics or New Relic, are targeting this microservices market. I think Honeycomb Enterprise can have something very dedicated for microservices because there is an explosion in the migration from monolithic to microservices. If Honeycomb Enterprise can create a stable solution which is easy to use and which gives additional value and helps for faster debugging with microservices, they can certainly gain market share from others.

Tracing is already there. I just wish that these tools are a bit less cryptic. These tools sometimes get quite cryptic for new users. The less cryptic they can be made, that can help these tools. Another thing is that for microservices, when you have multiple microservices installed, that is also required. There are tools where you install on a single microservice, but then these microservices interact with multiple microservices. That kind of picture, I have seen that in AppDynamics; they do give a picture showing that a particular request which arrived here had interaction with these other third-party services or microservices and databases. That is what we need. That is what performance engineers and SREs need to see for each request, where it spent the entire time; how many other services or databases it interacted with and what took more or less time, and if there is a sequence, it should highlight that also. Was it parallel or if, for instance, a call to service A and then a call was made to a database, or a call to service A and a database were in parallel, that kind of information.

I have used Honeycomb Enterprise for around a year.

Another thing I want to add here is that the team here had tried to use Honeycomb Enterprise earlier for tracing, but they faced issues. They could not get proper tracing with Honeycomb Enterprise at that time. That is what I have been given as feedback. This was also one of the reasons the team is less interested in using Honeycomb Enterprise. Most recently, as far as I know, they started using an open-source tool now, Jaeger. They also brought in Jaeger because when I was using Honeycomb Enterprise, I got this feedback that they could not use Honeycomb Enterprise meaningfully in the team, although they had this license and everything. What I know is that it was crashing with the application. These tools, as they instrument into the application, they can sometimes lead to crashes. I had seen this with AppDynamics also, but that was for a different client. There, we could see AppDynamics did provide a fix for that. The team did follow up with AppDynamics. They had a wider setup of AppDynamics in the company, so they could not just get rid of that. A similar feedback I also got from the team that when they were trying to use Honeycomb Enterprise for tracing, it was causing issues.

As I said, there have been issues with Honeycomb Enterprise. The team had reported to me that at one point in time when they were trying to use Honeycomb Enterprise, they faced stability issues. It was interacting with the application and causing some crashes in the application.

I would rate the stability of Honeycomb Enterprise around five.

Frankly speaking, we did not use it at such a large scale here, so I cannot comment on that. The Datadog we use here is quite scalable. That is being used for at least eight thousand hosts. Given the cloud version and all, the scalability depends on our provider, the service provider who gave us the Honeycomb Enterprise license. That is something which is beyond my scope.

The setup sounds a bit complex. The overall setup at our client's end is really very complex because they have high security limitations, which makes it a bit difficult to use any of the tools.

When other organizations are evaluating their APM tools, I could name Honeycomb Enterprise as one of the APM tools, another addition. There are not that many APM tools, with the top ones being Dynatrace, AppDynamics, New Relic and a few others. I heard about Honeycomb Enterprise in my current project only; I did not hear about Honeycomb Enterprise before. Given the new technologies emerging, if Honeycomb Enterprise can provide better traceability and monitoring, they have a chance.

Another thing is that for microservices, when you have multiple microservices installed, that is also required. There are tools where you install on a single microservice, but then these microservices interact with multiple microservices. That kind of picture, I have seen that in AppDynamics; they do give a picture showing that a particular request which arrived here had interaction with these other third-party services or microservices and databases. That is what we need. My overall rating for Honeycomb Enterprise is five out of ten.

My main use case for Honeycomb Enterprise is alerting and full stack tracing. I use Honeycomb Enterprise for tracking our traces from the front end all the way to our back end in Kotlin, and through Kafka and into Elastic. This helps us to spot any bottlenecks and see where errors occurred in our data processing pipeline.

We primarily use Honeycomb Enterprise to power our alerting, so we get alerts based on throughput, based on spikes, anomalies, and any error rates in each service. The best features Honeycomb Enterprise offers are alerting, which is mainly what we use it for.

Visualization is fine for me, although our dashboards are a bit cluttered. Primarily, the alerts are the most useful thing. The querying is good, although sometimes a bit messy. I would appreciate a better way to store my queries, perhaps with some auto-fill functionality. The alerts are the best feature.

Honeycomb Enterprise has positively impacted our organization by providing live alerts. We have been able to get alerts when something may pop up and see any issues in the system. We get alerts into Slack, and they work great. We see a lot of metrics go through into Slack, and they are really useful for keeping our team focused on only seeing one place to see alerts.

Honeycomb Enterprise can be improved by having a cleaner dashboard and a nicer way to search between insights.

I have been using Honeycomb Enterprise for one and a half years.

I do not have the metrics for a return on investment, but it has been very useful.

Having those alerts and metrics in Slack helps with our SLAs. If we ever have downtime in a service, we can get back to it straight away. I rate Honeycomb Enterprise a seven out of ten because I feel a lot of the journeys could be made cleaner. Some of the dashboards and the insights could be a bit more well-refined. For example, Grafana has a lot better visualization tools, but it needs to be a bit more involved.

Honeycomb Enterprise is deployed in our organization in a public cloud and is hosted by Honeycomb, who is the provider.

My experience with pricing, setup cost, and licensing is that the number of samples would be good. The number of events, really, so we can pay less. I do not have the metrics for a return on investment, but it has been very useful. It is quite expensive, but I think we get value out of it. My overall rating for Honeycomb Enterprise is seven out of ten.

We were building a product for one of the biggest wealth management platforms in the world, an American wealth management platform. For them, it is really important for the product to be reliable and for them to set up KPIs, especially for vendors like us who worked for them.

The debugging process usually involved Splunk Cloud or Honeycomb Enterprise traces. Whenever I was looking at an issue, I probably went through the traces because it was a microservice architecture. Sometimes it really helped to understand the call chain. For example, if there were 10 microservices calling each other in some sort of order, being able to visualize that and look through that was pretty useful.

I would say all of these three are pretty good features of Honeycomb Enterprise.

Honeycomb Enterprise is super useful if you think before this paradigm shift, when it was really important for humans to be able to see things visualized. For me to better understand in this very complex microservice architecture what's going on, Honeycomb Enterprise really helped with that through its good UI. However, the reason it's only five is because it's lagging behind in terms of AI-compatible features.

The major thing that's missing from Honeycomb Enterprise is AI compatibility. As far as I know, it's not really a text-based or code-based tool. It's more of a UI right now, which before this paradigm shift where everyone is using AI agents to work, was pretty useful. However, after this paradigm shift, I think it's really important for a tool like this to be AI-friendly. Honeycomb Enterprise is really lagging behind in this area, and I don't know how they could manage that.

For us it was sometimes pretty slow using Honeycomb Enterprise. I don't know if that can be improved. Although we might use the paid solution or self-host it somewhere because of privacy concerns. Maybe that's not Honeycomb Enterprise's fault. However, the main thing is that I think everything should very hard aim for the direction of being AI compatible because every engineer, or most engineers now use AI to code. If something is not easy to work with AI agents, that will stay in the past.

I used Honeycomb Enterprise at my previous job for around one to one and a half years.

Honeycomb Enterprise is stable.

It's very scalable since we used it for a really big organization and it worked.

If it's a big company, I understand using Honeycomb Enterprise. If it's a small company, I would suggest using some sort of open-source solution or something that is code-based and is easier to use with AI.

Although Grit is a tool code code migration and management of technical debt for large chunks of work, we reviewed Grit from the use case of assisting in faster remediation of vulnerable libraries. We examined 3 areas and how we could use the synergy of Grit.io along with Snyk.io that helps overcome Snyk's limitations:

1. Deep scanning and reachability analysis

2. Management of auto-generated Pull Requests (PRs)

3. Reduction of false positives

I'm connected and had interactions with the founder Mr. Morgante Pell, while I designed a comprehensive synergistic solution, and I wrote a 35+ page technical paper on this topic.

Large organizations with hundreds of development teams and tens of thousands of code repositories face challenges in efficiently identifying and remediating potential vulnerabilities within third-party libraries across numerous projects. Manual scanning and updating is time-consuming, error-prone, and can lead to delays in addressing security risks.

While Grit.io is not primarily a vulnerability scanner, its pattern-matching and code transformation capabilities can be adapted for mass identification and remediation of vulnerable libraries.

For each of these 3 areas listed above, we examined how Grit.io's unique features can complement Snyk.io's capabilities, resulting in a more robust and efficient security scanning process. We realize This synergistic approach addresses the limitations of relying solely on Snyk.io, resulting in improved code security and reduced risk of overlooking critical vulnerabilities.

The limitations of security scanning tools like Snyk.io represent real challenges faced by development teams on a daily basis. These limitations can lead to:

- Missed vulnerabilities in complex code structures

- Overwhelming numbers of auto-generated PRs, causing developer fatigue

- High rates of false positives, leading to wasted time and resources

We considered implementing Grit into our pipelines to address these specific scenarios for code security, though Grit isn't a security tool:

- Custom Rules and Pattern Creation

- Remediation Pattern Creation

- Automated Code Updates

- Custom Pattern Recognition

- Pull Request Generation

- and others

1. Grit.io's flexibility allows for custom rules and patterns to identify vulnerable libraries, extending its use beyond traditional refactoring tasks.

2. Automated pull requests streamline the remediation process, facilitating efficient mass updates across multiple repositories.

3. While not a replacement for dedicated security tools, Grit.io can be a valuable addition to a large organization's security toolkit for vulnerability identification and remediation.

4. The approach offers significant benefits in terms of efficiency, consistency, and proactive security management, particularly valuable for organizations with large, distributed development teams.

I asked very specific questions to Mr. Pell about consideration of code security scenarios in pattern design and rules, specifically that tuned with OWASP Top 10. I believe addition of code security focus can be a value-add, though the way Grit architecture is designed and how it works, it is and may not become an alternative choice of code security solutions. Rather, it must be treated as a powerful supplementary tool that augments the existing code security solutions (such as Snyk or Checkmarx) in a DevSecOps or Secure DevOps environment.

Anyone interested in learning more on this front or have queries, can get in touch with me for a consulting.

Our internal comprehensive evaluation of Grit spans over 6 months to a year since our client organization considered Grit under the Accelerator program of promising AI startups back in Sep 2023. Different phases of the implementation have been conducted by various development architects spanning several scenarios. Our scenario was very specific to how Grit's AI-powered capabilities could be leveraged on code security remediations for a large tech ecosystem.

The solution is mainly used for stack observability. It observes service behavior or any kind of failure that may be happening. The tool is also related to research. My company is working more on this, but I have been working on my SLOs and defining SLOs for the last seven months.

The solution's most valuable features are the queries for the OpenTelemetry events and all the tracing. The solution is very easy to use, and the dashboard is very intuitive.

We faced some OpenTelemetry metrics lost between the communication from the service and the Honeycomb.io. I can't say if this is a Honeycomb.io issue or if there are some limitations in OpenTelemetry.

Alerts are very helpful in Honeycomb.io, but we don't usually merge because we can compare queries with queries for making alerts. We can make alerts based on static numbers, which may block us from building alerts that could be generic enough or could be serviced.

I have been using Honeycomb.io for two years.

We’ve never had any issues with the solution’s stability.

Honeycomb.io is a scalable solution. The service is very resilient and can handle a lot of data. The quantity of data Honeycomb.io can parse and use to create charts is really good. More than 100 users are using the solution in our tech team.

I rate the solution’s scalability an eight or nine out of ten.

The solution's initial setup is easy. I think the hardest part is to understand OpenTelemetry in general.

We set up Honeycomb.io on all the services so that we can have all the set traces of the communication between all the services inside the company. This helps us understand where it could be failing, which in turn helps with failures and observability.

When we are not thinking about one specific failure but a major one, we can create queries for statistics views like the P99 or P95 behaviors. That's very helpful. I would recommend the solution to other users because it is very helpful.

Overall, I rate the solution a nine out of ten.