Arize AI

What do you like best about the product?

I really like how Arize AX makes it easier to understand what's happening with our AI. The tracing, monitoring, and evaluation tools are especially valuable to me. Tracing is particularly useful since it lets me look at individual AI interactions in more detail and understand what happened when a response didn't perform as expected. I also appreciate that I don't need to be extremely technical to get useful information from it, which is important for me.

What do you dislike about the product?

The learning curve. Arize AX has a lot of functionality, which is great, but it can sometimes feel a little overwhelming when you're first getting familiar with the platform, especially if you're not coming from a highly technical background. The onboarding could be improved by making it more guided and role-based. For example, someone using Arize AX from a product support perspective probably doesn't need the same level of technical details upfront as an engineer or data scientist.

What problems is the product solving and how is that benefiting you?

Arize AX helps me understand AI performance and identify issues without manual data digging. It saves time troubleshooting and provides insights to improve user experience.

What do you like best about the product?

What I like best about Arize Ax is the end-to-end visibility it provides into AI applications. You can trace an agent's workflow, quickly identify where something went wrong, and then use evaluations and experiments to understand whether a prompt, model, or workflow change actually improves performance. Having observability, evaluation, and experimentation together makes the feedback loop much faster than relying on logs or separate tools. In short, the best part is how easy it makes debugging AI systems. I can go from seeing that an agent is failing to tracing exactly what happened and evaluating the issue.

What do you dislike about the product?

The LLM/AI observability and evaluation platform has its drawbacks. My main criticism would be that if you only need basic LLM tracing, logging, and a handful of evaluations, Ax might be more infrastructure than you need. There are many concepts like traces, spans, datasets, experiments, evaluation, annotation, and Phoenix/AX terminology, which can take time to internalize. Once you move beyond standard evaluation into custom domain-specific evaluation, you may spend significant effort designing dataset prompts, scoring logic, and analysis workflows.

What problems is the product solving and how is that benefiting you?

Arize AX essentially solves the problem of making AI/LLM systems observable, debuggable, and reliable in production. When companies deploy AI agents and LLM applications, it can be difficult to answer traditional application monitoring, as it often isn't enough because an AI response can fail without the application itself technically crashing. Arize AX provides observability across the AI/agent workflow, including traces, model/LLM behavior, prompts, tool calls, retrieval, RAG steps, evaluation, latency, and costs, so teams can identify and fix problems faster.

What do you like best about the product?

The platform is also useful for teamwork because it gives developers, data scientists, and AI teams a shared place to understand how their applications are performing. Overall, I like Arize AX because it provides better visibility, makes AI problems easier to find, saves time during testing and debugging, and helps teams build AI applications that are more reliable and useful. I also like that it can help teams improve AI systems over time.

What do you dislike about the product?

The main thing I dislike about Arize Ax is that it can feel a bit difficult for new users when they first start using it. There are many features, settings, dashboards, and options, so it may take some time to understand everything properly. The platform is powerful, but sometimes the amount of information shown can feel overwhelming at first.

What problems is the product solving and how is that benefiting you?

Arize AX is helping solve many problems related to building, testing, and improving AI applications. One of the biggest challenges is that it can be difficult to understand why an AI model gives a wrong, poor, or unexpected answer. Arize AX makes it easier to test AI applications and see where issues are happening.

Arize AX also helps with debugging. When an AI response is not correct, it can be hard to pinpoint the exact reason. The platform provides better visibility into the AI process, which makes troubleshooting and improvement more straightforward.

What do you like best about the product?

What I like most about Arize AX is how helpful it is for monitoring AI workflows and spotting issues early. It gives me a clear view of how my models are performing, and it makes it easier to troubleshoot when I run into unexpected outputs.

What do you dislike about the product?

The biggest downside is that it can feel a little overwhelming at first, especially when you’re looking at a lot of metrics and traces. It also takes some time to set everything up and configure it in a way that’s genuinely useful for my specific workflow.

What problems is the product solving and how is that benefiting you?

Arize AX helps me understand what’s going on with AI models and workflows without having to manually dig through outputs. It makes it easier to catch unexpected results, troubleshoot problems, and track performance over time. That saves me time and gives me more confidence in the AI tools I rely on for my work.

What do you like best about the product?

I like how straightforward Arize AX provides direct visibility into AI application traces, making it easier to understand what is happening at each step of request

What do you dislike about the product?

I feel the interface can become a bit overwhelming when there is a lot of information to go through, especially for complex workflows

What problems is the product solving and how is that benefiting you?

It is solving the problem of understanding and monitoring what is happening inside AI applications, very useful in production scenarios

What do you like best about the product?

I like how easily I can trace issues through LLM and agent workflows

What do you dislike about the product?

The cost can become significant as usage and data volume increase

What problems is the product solving and how is that benefiting you?

Give me visibility into model quality and performance so I can make improvements faster

Recommendations to others considering the product:

Consider optimizing data usage to manage costs effectively.

What do you like best about the product?

I like the visibility it provides into AI application performance. The dashboards make it easier to look at traces and understand where an issue may be happening. The evaluation and monitoring features are also useful for getting a better idea of how the application is performing over time.

What do you dislike about the product?

There are quite a few features to explore, so it takes some time to understand the platform properly. Some of the more advanced options may also require a bit of learning before they become useful.

What problems is the product solving and how is that benefiting you?

It provides a centralized place to monitor and evaluate AI applications. Instead of relying only on application logs, it gives more visibility into traces and performance, which can make troubleshooting and quality checks easier.

What do you like best about the product?

I really like the end-to-end visibility Arize AX provides into AI applications. The tracing feature is incredibly useful because it allows me to drill into an individual request and see the entire workflow, which is much better than guessing from application logs. I also appreciate the evaluation and monitoring capabilities, particularly when we're testing changes to prompts or models. It's handy for quickly identifying when a new prompt starts producing more irrelevant responses by comparing results. The interface is fairly easy to navigate once I got the hang of how the data is organized, which adds to the overall ease of use.

What do you dislike about the product?

The biggest improvement area for me is the initial setup and learning curve. Getting instrumentation, traces, and evaluations configured properly can take some time, especially with an existing application. Also, when there's a lot of trace data, finding the exact issue can sometimes require multiple filters and views. A simpler troubleshooting workflow would make it more efficient. A few things would help: smarter filtering and search, saved troubleshooting views, and better grouping of similar traces. For example, being able to quickly filter by model version, error type, latency, or a specific prompt would save time. An AI-assisted summary of recurring trace issues would also be useful when we are dealing with a large volume of data.

What problems is the product solving and how is that benefiting you?

I use Arize AX for monitoring AI/ML applications in production. It provides visibility into model performance, helps identify issues quickly, and makes troubleshooting much easier by tracing requests and analyzing outputs.

What do you like best about the product?

- One platform that covers the full loop: tracing, datasets, evaluation, and experiments
- Evaluation is a first-class product, not an afterthought
- Broad framework instrumentation that follows OpenInference standards
- Strong CLI that makes evaluation and experimentation reproducible and scriptable
- Tight integration between live trace data, human annotation, and LLM-as-judge scoring

What do you dislike about the product?

- Steep setup: column mappings must match real span paths, which is easy to get wrong
- Index lag (1-2 hours for evals, 6-12 hours for time-series) makes recent data slow to appear
- Custom code evaluators are brittle and fail silently (hard to debug)
- Documentation can be inconsistent between versions
- Requires careful planning before it feels smooth

What problems is the product solving and how is that benefiting you?

Problems solved: I didn’t have clear visibility into LLM app behavior across prompts, tools, and model calls, and it was hard to tell when model quality regressed in production. Evaluation and experimentation were spread across disconnected tools, with no single source of truth for traces, datasets, and eval scores.

Benefits: This gives me one platform for tracing, evaluation, and experiments, which makes prompt optimization more data-driven instead of guesswork. I also like having reproducible evaluation workflows via the CLI, and it speeds up root-cause analysis when digging into trace data.

What do you like best about the product?

My favorite part is being able to see how an AI agent moves through different steps which makes complex workflows much easier to troubleshoot

What do you dislike about the product?

Some workflows require a good understanding of instrumentation and evaluation concepts so getting everything configured correctly can take time

What problems is the product solving and how is that benefiting you?

Arize AX helps connect development and production monitoring so I can test changes, measure performance, and make improvements based on real application data