Honeycomb Enterprise

We were building a product for one of the biggest wealth management platforms in the world, an American wealth management platform. For them, it is really important for the product to be reliable and for them to set up KPIs, especially for vendors like us who worked for them.

The debugging process usually involved Splunk Cloud or Honeycomb Enterprise  traces. Whenever I was looking at an issue, I probably went through the traces because it was a microservice architecture. Sometimes it really helped to understand the call chain. For example, if there were 10 microservices calling each other in some sort of order, being able to visualize that and look through that was pretty useful.

I would say all of these three are pretty good features of Honeycomb Enterprise .

Honeycomb Enterprise is super useful if you think before this paradigm shift, when it was really important for humans to be able to see things visualized. For me to better understand in this very complex microservice architecture what's going on, Honeycomb Enterprise really helped with that through its good UI. However, the reason it's only five is because it's lagging behind in terms of AI-compatible features.

The major thing that's missing from Honeycomb Enterprise is AI compatibility. As far as I know, it's not really a text-based or code-based tool. It's more of a UI right now, which before this paradigm shift where everyone is using AI agents to work, was pretty useful. However, after this paradigm shift, I think it's really important for a tool like this to be AI-friendly. Honeycomb Enterprise is really lagging behind in this area, and I don't know how they could manage that.

For us it was sometimes pretty slow using Honeycomb Enterprise. I don't know if that can be improved. Although we might use the paid solution or self-host it somewhere because of privacy concerns. Maybe that's not Honeycomb Enterprise's fault. However, the main thing is that I think everything should very hard aim for the direction of being AI compatible because every engineer, or most engineers now use AI to code. If something is not easy to work with AI agents, that will stay in the past.

I used Honeycomb Enterprise at my previous job for around one to one and a half years.

Honeycomb Enterprise is stable.

It's very scalable since we used it for a really big organization and it worked.

If it's a big company, I understand using Honeycomb Enterprise. If it's a small company, I would suggest using some sort of open-source solution or something that is code-based and is easier to use with AI.

Although Grit  is a tool code code migration and management of technical debt for large chunks of work, we reviewed Grit  from the use case of assisting in faster remediation of vulnerable libraries. We examined 3 areas and how we could use the synergy of Grit.io along with Snyk .io that helps overcome Snyk 's limitations:

1. Deep scanning and reachability analysis

2. Management of auto-generated Pull Requests (PRs)

3. Reduction of false positives

I'm connected and had interactions with the founder Mr. Morgante Pell, while I designed a comprehensive synergistic solution, and I wrote a 35+ page technical paper on this topic.

Large organizations with hundreds of development teams and tens of thousands of code repositories face challenges in efficiently identifying and remediating potential vulnerabilities within third-party libraries across numerous projects. Manual scanning and updating is time-consuming, error-prone, and can lead to delays in addressing security risks.

While Grit.io is not primarily a vulnerability scanner, its pattern-matching and code transformation capabilities can be adapted for mass identification and remediation of vulnerable libraries.

For each of these 3 areas listed above, we examined how Grit.io's unique features can complement Snyk.io's capabilities, resulting in a more robust and efficient security scanning process. We realize This synergistic approach addresses the limitations of relying solely on Snyk.io, resulting in improved code security and reduced risk of overlooking critical vulnerabilities.

The limitations of security scanning tools like Snyk.io represent real challenges faced by development teams on a daily basis. These limitations can lead to:

- Missed vulnerabilities in complex code structures

- Overwhelming numbers of auto-generated PRs, causing developer fatigue

- High rates of false positives, leading to wasted time and resources

We considered implementing Grit into our pipelines to address these specific scenarios for code security, though Grit isn't a security tool:

- Custom Rules and Pattern Creation

- Remediation Pattern Creation

- Automated Code Updates

- Custom Pattern Recognition

- Pull Request Generation

- and others

1. Grit.io's flexibility allows for custom rules and patterns to identify vulnerable libraries, extending its use beyond traditional refactoring tasks.

2. Automated pull requests streamline the remediation process, facilitating efficient mass updates across multiple repositories.

3. While not a replacement for dedicated security tools, Grit.io can be a valuable addition to a large organization's security toolkit for vulnerability identification and remediation.

4. The approach offers significant benefits in terms of efficiency, consistency, and proactive security management, particularly valuable for organizations with large, distributed development teams.

I asked very specific questions to Mr. Pell about consideration of code security scenarios in pattern design and rules, specifically that tuned with OWASP Top 10. I believe addition of code security focus can be a value-add, though the way Grit architecture is designed and how it works, it is and may not become an alternative choice of code security solutions. Rather, it must be treated as a powerful supplementary tool that augments the existing code security solutions (such as Snyk or Checkmarx) in a DevSecOps  or Secure DevOps environment.

Anyone interested in learning more on this front or have queries, can get in touch with me for a consulting.

Our internal comprehensive evaluation of Grit spans over 6 months to a year since our client organization considered Grit under the Accelerator program of promising AI startups back in Sep 2023. Different phases of the implementation have been conducted by various development architects spanning several scenarios. Our scenario was very specific to how Grit's AI-powered capabilities could be leveraged on code security remediations for a large tech ecosystem.

The solution is mainly used for stack observability. It observes service behavior or any kind of failure that may be happening. The tool is also related to research. My company is working more on this, but I have been working on my SLOs and defining SLOs for the last seven months.

The solution's most valuable features are the queries for the OpenTelemetry events and all the tracing. The solution is very easy to use, and the dashboard is very intuitive.

We faced some OpenTelemetry metrics lost between the communication from the service and the Honeycomb.io. I can't say if this is a Honeycomb.io issue or if there are some limitations in OpenTelemetry.

Alerts are very helpful in Honeycomb.io, but we don't usually merge because we can compare queries with queries for making alerts. We can make alerts based on static numbers, which may block us from building alerts that could be generic enough or could be serviced.

I have been using Honeycomb.io for two years.

We’ve never had any issues with the solution’s stability.

Honeycomb.io is a scalable solution. The service is very resilient and can handle a lot of data. The quantity of data Honeycomb.io can parse and use to create charts is really good. More than 100 users are using the solution in our tech team.

I rate the solution’s scalability an eight or nine out of ten.

The solution's initial setup is easy. I think the hardest part is to understand OpenTelemetry in general.

We set up Honeycomb.io on all the services so that we can have all the set traces of the communication between all the services inside the company. This helps us understand where it could be failing, which in turn helps with failures and observability.

When we are not thinking about one specific failure but a major one, we can create queries for statistics views like the P99 or P95 behaviors. That's very helpful. I would recommend the solution to other users because it is very helpful.

Overall, I rate the solution a nine out of ten.