Fiddler AI (Lite Version)

My main use case for Fiddler AI is as a tool to monitor, explain, and govern machine learning models and generative AI applications, which helps organizations build trust by tracking their performance, preventing security threats like hallucination and prompt injection, and ensuring regulatory compliance.

In a recent project and task, I showcase how we use Fiddler AI's key features like AI observability and monitoring, which track hundreds of metrics across traditional machine learning models and large language models to monitor performance, data drift, and accuracy. We use it as security and guardrails that implement real-time safety measures to block harmful outputs and protect against jailbreak and prompt injection attempts. We also utilize it for explainability, helping teams understand why an AI model made a specific prediction or decision, providing deep root cause analysis when problems occur. Lastly, we employ it for governance and compliance, creating centralized audit trails and model cards to help companies meet strict regulatory standards and risk management requirements.

Fiddler AI has significantly impacted my organization positively, particularly in security and guardrails. As we use many tools, we previously had humans monitoring all of them. Human errors created loopholes, but now with Fiddler AI, which is trained using our company policies and playbooks, human intervention has become minimal. We have reduced human intervention by 70% and human mistakes by around 90%, which are the points I wanted to mention.

Reducing human intervention by 70% directly affects our productivity and day-to-day work. A person who worked with one policy or guardrail for a week now hardly takes one day, saving us six days of effort for each cycle. Moreover, the reduction in human mistakes has significantly improved product quality, with great feedback from vendors and other businesses using the product, increasing confidence levels and scores when using Fiddler AI.

I have observed one area where Fiddler AI can be improved. When explaining a product, it is essential to specify company policies and guardrails. Without providing company-specific criteria, the product may hallucinate, as it will compare international policy boards with our company's policies. Therefore, to achieve better results, we need this improvement.

While support for Fiddler AI is quite good, documentation needs improvement. The tool's documentation should specifically mention the policies, security aspects, and all organization-specific documents alongside generic documents to provide more precise information.

I have been using Fiddler AI for the past two years.

Fiddler AI is stable and consistent in its performance for us.

Its scalability is excellent and can handle growing workloads very easily and effectively.

Customer support for Fiddler AI is very good. They support critical issues (P1) within a day or a few hours, while medium or low priority issues (P2) are fixed within two to three days.

Before choosing Fiddler AI, we evaluated several options, including Gemini, ChatGPT, Azure, and other vendor products. Ultimately, we found Fiddler AI to be the best tool for security and governance purposes in the market.

I have seen a return on investment with Fiddler AI. The investment reflects both quality improvements and savings in time and money. Previously, we used a manual solution, requiring us to validate every security aspect manually before deployment. Now, with this AI solution, we have increased efficiency. For instance, what used to take 12 months for validation is now accomplished much faster, directly benefiting our costs.

Regarding the features, I want to mention that top AI products are generally split into two categories: versatile conversational assistants and specialized creation tools. Widely adopted and highly rated AI products include ChatGPT, Claude, Google Gemini, and others. However, Fiddler AI focuses on governance and compliance, and security and guardrails, which is why I respect the tool and see many use cases for it.

Among the features I mentioned, I find governance and compliance the most valuable and impactful in my work because, in our IT company, we use many tools daily. As part of our day-to-day activity, we utilize in-house and vendor-specific tools. If the governance and compliance section is maintained by AI, it highlights those features, which is significant for the Information Resource Management (IRM) team to get details on governance and compliance. We use hundreds of vendor-specific tools, which makes it hard to measure governance or regulatory aspects every time. Fiddler AI checks this holistically and provides a dipstick report on the same governance and compliance.

Fiddler AI is deployed in our organization first in UAT and then in production. We use AWS as the cloud provider for deployment. We purchased Fiddler AI through the AWS Marketplace.

Regarding Fiddler AI's capabilities, I think its governance and security are very high, featuring two to three layers of security that provide excellent protection for secure data. The accuracy and reliability of Fiddler AI's output range between 70% to 80%, and I can trust it. However, I believe that 20% to 30% requires human check or validation, as AI's outputs need checking for correctness.

My advice for others looking into using Fiddler AI is that if a company or individual seeks security, governance, and scalability for their product, they should definitely consider it. I would rate this review an 8 out of 10.

Public Cloud

Amazon Web Services (AWS)

We primarily use Fiddler AI to monitor our AI models in production and catch issues like drift or unexpected prediction changes early. For example, if a client-facing model starts behaving differently after new data comes in, we use Fiddler AI to trace the problem, understand why it happened, and decide whether the model needs adjustment or retraining.

As a web agency, we have been using Fiddler AI for about eight months now, and it has proven to be excellent. Fiddler AI fits into our workflow as an early warning and debugging layer for production models. It helps us spend very little time manually investigating issues and allows us to focus more time on fixing the actual root cause, which makes our model operations much smoother. Over time, it has become part of our regular workflow whenever we need to check the model health or investigate unexpected behavior. Instead of starting from scratch, we can quickly narrow down to the issue and move faster on resolution with the help of Fiddler AI. For us, it is not just a monitoring tool but rather part of how we keep our AI systems reliable every day.

The two features we rely on most are drift detection and explainability. Drift detection helps us spot issues early, and explainability helps us understand the root cause quickly so that we can take action faster. Together, those features make it easier to monitor and debug models in production.

The alerting feature is also helpful with Fiddler AI because it lets us respond quickly when something changes. The interface makes it easy to move from an alert to a specific model or data slice that needs attention. This saves a lot of time and keeps the workflow smooth.

What helps most with Fiddler AI is the combination of drift detection and explainability. We can first see when the performance or prediction patterns have changed, and then drill into the specific features or slices of data causing the issue. That makes it much easier to figure out whether the problem is data drift, a bad input pattern, or something in the model logic. The most valuable aspect is that Fiddler AI shows both the change and the reason behind it, so we can trace the issues from an alert into the exact data slice or feature that caused the problem.

The main thing I would improve with Fiddler AI is onboarding and ease of use for different stakeholders. The platform itself is very powerful, but making some workflows more guided and adding more ready-made dashboards would make it even better for a team in our situation. It is strong overall, but simpler onboarding, better dashboards, and more automation would make it even easier to use.

I have been working in ADP for about two years and six months to be precise.

Fiddler AI has been stable for us overall, and we have not run into any downtime or reliability issues. It has been dependable for day-to-day monitoring and investigations, and I commend the team managing it.

Fiddler AI has scaled very well for us so far. It has been able to handle our workload even better as we add more models because it gives us a centralized way to monitor and debug everything. From what we have seen, it scales very effectively, and as our usage grows, it still feels manageable and does not become a bottleneck for the team.

We have interacted with Fiddler AI's support team very rarely, but the experience has been very good. They are very responsive, professional, and helpful when we needed clarification or guidance. The support overall was excellent.

We were using a more manual approach before Fiddler AI, along with some basic monitoring tools. We moved to Fiddler AI because it gives us much greater visibility into model behavior and makes debugging much faster.

The documentation, integrations, and everything else look perfect. The only suggestion would be if they could add simpler onboarding, better dashboards, and more automation, as it would be even better.

I am satisfied with Fiddler AI, and I have discussed everything with the relevant parties.

We have clearly seen the return on investment mainly through the time saved and more efficient use case of our eight-member team. We spend much less time manually investigating model issues, which has become a very significant advantage. This means fewer hours lost on debugging and faster incident response. We have not had to add extra headcount just to manage model monitoring, so the platform has helped us stay efficient as we scale.

Pricing for Fiddler AI feels appropriate for an enterprise-level product, so it is not the cheapest option, but the value is there for a production AI use case. The setup cost was manageable since it was a hosted platform, and licensing was straightforward for a team-based deployment.

We did look into a few other options before choosing Fiddler AI, mostly general ML monitoring and observability tools, as well as our internal manual approach. Fiddler AI stood out because it was a strong option on explainability, drift detection, and making production debugging easier.

My advice to others looking into using Fiddler AI would be straightforward. If you need serious production monitoring and faster model debugging, go with Fiddler AI. It is especially valuable if your team wants better visibility into drift, explainability, and incident response. I recommend starting with a clear workflow and making sure your team is aligned on what to monitor so that you get the value from it very quickly. I have given this review an overall rating of 8.

Public Cloud

Amazon Web Services (AWS)

I am currently working with D Innovation Private Limited for a client called Casia in a Senior Data Engineer profile, where I have been working for the last 1.6 years. I have a total of eight years of experience working at Infosys as well.

I use data engineering tools for Power BI and Fiddler. For analysis, I have to create a model to analyze the data. I utilize our data for this purpose.

In day-to-day work, if I need to support or create any language-based model with features, I find it very easy to use. I create modular solutions that convert from one language to another. Fiddler AI is the best way to handle this right now.

I am utilizing Fiddler AI for AI agents with multiple Matillion tools and Kubernetes. I mention that we have AI agents for these purposes. The pricing is good, at approximately 0.002 dollars per developer, which is very cost-effective. Additionally, the latency is less than 100 microseconds, which is excellent for handling Fiddler AI.

I have been using Fiddler AI in multiple ways for approximately two to three years.

Fiddler and Fiddler AI serve different purposes. I utilize Fiddler for creating Power BI reports, and it is very good visually. There are multiple options for color choices and data design, allowing me to explain data effectively. Right now, I utilize Fiddler AI for machine learning purposes.

In Power BI projects, I use it for data enhancement and data quality checking for schema validation. I use it for data analysis as well. I connect it directly to my reports through Fiddler API and write Python code for analysis. I integrate this into my Power BI projects.

All features in Fiddler AI are good, offering excellent connectivity options. The REST API is good, supporting multiple languages with connected speech and responsive answers for any AI tools. Data performance is very good, and it is very easy to use. From a learning perspective, it provides responsive answers for data manipulation and other utilization needs.

To improve things, I need to connect it for handling LLM prompts and data management. This is mainly for understanding queries better. Snowflake Cortex is similar, but it does not give very good responses and sometimes provides poor experiences. Fiddler AI provides a better way.

I should not give too much data to any AI agent because when an agent reads and analyzes data itself, this is the wrong approach. Giving full data access to any organization is not a good practice. I need to analyze any tool first before proceeding. After that analysis, I can move forward.

I have been using this solution for approximately six to seven years.

Fiddler AI is stable and performs well.

Scalability is very good. The solution handles 7 to 15x more data, which is a very good opportunity.

Customer support is very good. We handle data and data breaks well, and Google Cloud support is available anywhere, which is excellent. Fiddler AI supports healthcare, AI defense, intelligence, healthcare data, and insurance data.

I did not use a previous solution and have just now tried Fiddler AI.

The initial setup is fine with no issues.

The implementation team was fine.

A significant amount of time is saved, which is very good for us. Both time and money are saved. If I needed to utilize a task with 10 people, I can now handle data analysis with only five people, which is very good for our organization.

Licensing cost is very low, which is very good. The usage cost is reduced by 70 to 80x compared to other solutions, and the latency is less than 100 microseconds, which also provides a very good value.

I have definitely considered other solutions.

I will need to check on further recommendations after some time. I rate this product at 8.5 out of 10.

Public Cloud

Amazon Web Services (AWS)

What do you like best about the product?

I had a really great experience using Fiddler AI. The platform is very easy to use, even for beginners, and the interface is clean and well-organized. What I liked the most is how accurately it analyzes data and gives clear insights without making things complicated. The performance is fast, and the results feel reliable, saving a lot of time compared to doing everything manually. I also appreciate how smoothly everything works without glitches. Fiddler AI is a powerful and user-friendly tool that I would definitely recommend to anyone looking for smart AI solutions. I like its simplicity and clarity, as it makes complex AI model insights very easy to understand with a clean interface and well-structured dashboards. I really appreciate how it provides real-time monitoring and highlights issues like errors or bias quickly, which saves a lot of time. Another thing I like is the reliability—it gives consistent and useful insights that help in making better decisions. Overall, it combines powerful features with ease of use, which makes it stand out.

What do you dislike about the product?

While Fiddler AI is very useful, there are a few areas that could be improved. Sometimes the platform can feel a bit complex for complete beginners, especially when exploring advanced features for the first time. A more guided onboarding or tutorials would help new users get comfortable faster. Also, the pricing can feel a bit high for individual users or small projects. Improving affordability or offering more flexible plans would make it accessible to a wider audience. Lastly, adding more customization options in dashboards and reports would make the experience even better.

What problems is the product solving and how is that benefiting you?

Fiddler AI helps me understand and trust AI model decisions by analyzing performance, detecting errors or bias, and monitoring clearly. It saves time through automated monitoring, giving reliable insights that improve accuracy and decision-making.