Tracing ai agents has boosted debugging, cut latency, and optimizes our support chatbot
What is our primary use case?
My main use case for
Comet is building AI agents, optimizing them, getting traces, and experimentation.
A specific example of how I use Comet for optimizing AI agents and tracing is with our chatbot called Oscar, which is a RAG-based, LLM-based chatbot where users can manage their chats, ask questions regarding the products, and it also acts as a support agent. Comet helps us log the traces, capture them, organize the LLM calls, and optimize the costing accordingly.
Comet has an inbuilt AI that helps us get conclusions for the cost of LLM calls and many other things.
What is most valuable?
The best features Comet offers in my opinion are the datasets and the prompt library, along with the evaluation. The traces are very deep, covering the tool calls and everything, so we have flow diagrams.
The flow diagrams help my team by aiding the log traces to capture and organize application calls, which helps in tracing from the first to last call if an API gets hit using LLM. We analyze how it works, how it travels, and each log trace using the flow diagram, which provides a visual representation.
Testing with multiple test datasets has helped us significantly.
Comet has positively impacted my organization as we have seen a drastic change. The product has improved since we started using Comet, and debugging has become much easier.
Debugging became easier for us because the tracing helps us in production to trace an LLM call, save the cost, reduce the tool calling, optimize the AI responses, increase the speed, and reduce the latency.
What needs improvement?
Comet can be improved by adding the MCP server to integrate with Chat GPT and other applications. If they provide skills to integrate while building applications to seamlessly incorporate Comet in AI agents, it would help significantly.
For how long have I used the solution?
I have been using Comet for within one year.
What do I think about the stability of the solution?
Comet is stable, and I have not faced any bugs as of now; it is going very well.
What do I think about the scalability of the solution?
Comet handles scalability well, accommodating growth and increased workloads.
How are customer service and support?
We have not reached out to customer support yet because we have not needed support as all things are going great.
Which solution did I use previously and why did I switch?
We previously used
Grafana, which was not meant for our purpose, so we felt Comet is better in terms of LLM Ops and tracing AI queries.
What was our ROI?
I have seen a return on investment since time has obviously been saved and productivity has improved. With the automation brought by Comet, the work we were doing with other tools has changed, so time has been saved and while we have not calculated the cost saved yet, it is likely there.
What's my experience with pricing, setup cost, and licensing?
My experience with pricing, setup cost, and licensing has been straightforward and easy to understand, offering unlimited team members and some metrics without needing customization for our plan yet.
Which other solutions did I evaluate?
Before choosing Comet, I evaluated
Sentry, but it is not more focused on AI; it is a combination of various logging and tracing software.
What other advice do I have?
My advice for others looking into using Comet is that if they are utilizing LLM bots at the enterprise level, tracing is essential, and Comet offers the best solution to trace logs and optimize the overall solution. I would rate this product a 9 out of 10.
Easy to Use !! Great UI
What do you like best about the product?
User interface
Easy to use
Support different View and Easy to search Text
What do you dislike about the product?
Price.
time take to pull data
small notification view
What problems is the product solving and how is that benefiting you?
Code Debug
Application monitoring
User point of view
What do you like best about the product?
It's provide the best ai interface in handling things. The user interface is excellent and all AI tools can be used abruptly with diffrebt build functions. It competes with the different launched ml in market.
What do you dislike about the product?
The comet.ml runs causly with the interruption with softwares , therefore sometimes slows down but it has ggod backup notion with carry forward improvements in consoles.
What problems is the product solving and how is that benefiting you?
AI related machine components are useful in solving the things both financially and end-ser type. The level of productivity increases with a diffrent level of business satisfication.
Easy to use and integrate
What do you like best about the product?
User-friendly interface for model training, easy to use, the dashboard makes visualizations convenient
What do you dislike about the product?
Somewhat limited model tracking abilities. Documentation could be improved.
What problems is the product solving and how is that benefiting you?
Used to manage end-to-end ML life cycle. Has helped streamline the process significantly and improved transparency within the team
Good Platform to share ML data
What do you like best about the product?
It provides a single platform to share ML experiments and models. Helps to compare data and insights from different people in an efficient way. This helps save a lot of time.
What do you dislike about the product?
It takes some time to ramp up on this. I wish that the documentation was more crisp or polished. A better adoption on the academic side would help students be familiar with this tool earlier.
What problems is the product solving and how is that benefiting you?
We use it to visualize our training data and it helps us understand the data and models better. Additionally, we don't have to spend time in talking to specific people since all data is available on the dashboard
Review ML program Comet.ML
What do you like best about the product?
The easy function to use and the platform is easy to access
What do you dislike about the product?
The complicated of the program and slowness
What problems is the product solving and how is that benefiting you?
The collaboration and reduce the time to work on the project
Comet ML a great tool for working with Machine Learning Models
What do you like best about the product?
Comet. Ml helps my team to track their ML models and also visualize them; the dashboards graphs provided by comet provide a detailed view of the model.
What do you dislike about the product?
There is no support for the R language. It's also a bit costly; It would be great to have it as open source.
What problems is the product solving and how is that benefiting you?
Comet. Ml helps my team being more productive by having a better analysis of models and compare and share results; also, it helps in analyzing underperforming models.
Comet ML for Data science Operations
What do you like best about the product?
The best thing about comet ml is I can integrate my developed machine learning model pipeline easily, and it provides great GUI.
What do you dislike about the product?
What i don't like about comet ml is it doesn't support for all the programming languages and need more details about all the features
What problems is the product solving and how is that benefiting you?
I have used Comet ML for tracking the Changes in my ML code, input files and parameters for model development in the operation phase
