Monte Carlo Data + AI Observability Platform

I work as a business analyst and I usually see data anomalies in our company's data set, and I also work a lot on Power BI reports to see our performance on the supplier side.

When we receive data from our suppliers to view their performance, sometimes the data is not complete or they are doing backfills of previous data, so we have established some rules in Monte Carlo to monitor these anomalies, and whenever we see something passing a preset limit, we receive an alert.

When I open Monte Carlo, I usually look at my dashboard to see how many alerts we have received in the last few days, but I usually check the alerts in the last month, and I see which rule has received the maximum amount of alerts; then I try to solve it first because the pattern is similar, and then I try to solve other alerts based on other rules.

In my team, it's me who handles those alerts, but we have another team who works on these alerts as well, although they are working on another kind of data set, but in the company, it's used by many people.

The best features of Monte Carlo for my work are the ability to see alerts clearly, how many alerts we have received on which rule and for which country, and there is a feature called investigation query inside Monte Carlo which shows a pre-done analysis, so you don't have to run an SQL query by yourself to do manual checks.

It gives a clear analysis in a large data set, which is very time-saving, so I don't have to run manual codes to verify the data, and it has helped me a lot in saving time and improving efficiency while doing data checks or verifying data. It's a really great tool to explain the anomalies when we see one in Monte Carlo, as we have actual proofs to show to people or to the managers that we are having this anomaly or that data is missing.

There's also an AI feature that is inbuilt in Monte Carlo, but you have to pay separately for that feature, and I used it for quite a while in the beginning, but now my company has disabled it.

The biggest pain point with Monte Carlo is that we have created some rules, but those rules cannot judge everything, and I think the platform is a bit complex for someone new, so it can be more intuitive; a display adoption platform could guide the user on how to use this, like a DAP system. It took a lot of time for me to learn it, and without a guide, a new user would be clueless.

If I could change one thing about Monte Carlo, it would be for the platform to suggest some data quality rules by itself or some algorithms based on the anomalies and the patterns of our anomalies, which would be helpful, and also changes in our rules according to past anomaly patterns. I think that would be good, and they should also improve their support system, as I find it a bit weaker at the moment.

It's been six months since I've been working on Monte Carlo, and it's a really great tool for analyzing data quality anomalies.

Previously, the data checks were performed manually; they extracted data on Snowflake and then did manual verifications on Excel using formulas.

It was not me who implemented Monte Carlo; it was another senior data analyst who implemented it a year ago, but I think it took a few months to get everything up and running.

It needed formal training because the tool is not that easy; if someone doesn't have a data analyst or business analyst background, you have to explain every rule which you have set by yourself, because the rules are created by us, not Monte Carlo; Monte Carlo is just a tool. We put our own rules to govern the data sets, and we literally had to make a guide to help users get to know that platform.

It was not me who implemented Monte Carlo; it was another senior data analyst who implemented it a year ago, but I think it took a few months to get everything up and running.

Monte Carlo saves me roughly 30% to 40% of my time in doing verifications or data quality checks, and related to my team, I'm the only one who uses it in my team, but we collaborate with another team that uses it.

I didn't ask that question in my company; it was a good choice, so it's a very popular tool, which is why my company picked Monte Carlo.

I would like to leave a review for the tool called Monte Carlo, which is a data quality analysis tool.

I think they just changed the layout a little bit, and during my work, I didn't see any changes in the platform except for the AI feature, which I don't use anymore.

Our final goal is to automate every data quality manual check into Monte Carlo so we don't have to do a lot of checks by ourselves.

If I find something bizarre, I go to Monte Carlo and see if it's happening there as well, so it gives me confirmation that the issue is occurring and we can contact our supplier or the country to verify what it is.

Monte Carlo is a really good tool; whenever I do data verification checks for the dashboards, and if I find something bizarre, I go to Monte Carlo and see if it's happening there as well, so it gives me confirmation that the issue is occurring and we can contact our supplier or the country to verify what it is. I would rate this product a 7 overall.

My organization relies on Monte Carlo for data observability, such as whether tables were loaded on time and whether the load met expectations regarding volume. We also use it for observability into our data transformation pipelines. We use it to trigger alerts to our respective product and engineering teams if loads are delayed or the volume of loads does not meet expectations. In essence, we use Monte Carlo to gain observability into our data.

We have encountered scenarios where a particular data load would not have generated any alerts if Monte Carlo had not been in place, since the data load went through successfully. However, the volume of the data loads in that case was below the threshold volume that triggered a volume anomaly alert from Monte Carlo. We were able to go back, fix the data, and report it to the upstream source.

We also use Monte Carlo to catch long-running queries. We have monitors set up for that purpose as well. We have an in-house solution developed to catch long-running queries on Snowflake in real time, something that is not currently available in any other SaaS providers for Snowflake.

The volume monitors and the anomaly monitors regarding volume, freshness, and data consistency are the best features Monte Carlo offers in my experience. These are the best features because they help flag issues that are more abstract and difficult to measure. While a data load that did not happen is an easier thing to track, a data load that happened but the volume was not in a particular range is a very tricky metric to monitor. That is a great feature.

It is mostly a combination of volume, freshness, and consistency monitors that I find myself relying on the most. The specific monitors I use depend on the business and use case we are catering to, the tables, and the data involved. It is difficult to point out one monitor as the most useful, but we use all three of them in different combinations very extensively.

Monte Carlo has had a major impact on my organization in terms of data health for the downstreams and for all the engineering teams that depend on our data. We are now getting timely alerts around the quality of the data, the volume of the data, and the health of the data. We are able to get that visibility more granularly into every single table. We are able to draw the data lineage to understand failures faster. Overall, Monte Carlo has had a very positive impact in terms of having healthier data and being able to trace through the data lineage to understand where exactly in the data life cycle things are going wrong.

Monte Carlo needs to stop their reliance on AI, as it is not going well and is degrading the entire product. They need to find their way back, establish a product roadmap, and have real engineers work on improvements rather than heavily push AI down users' throats. They need to stop relying on AI as heavily as they have been doing, as this has really degraded the user experience. The overall direction they are taking with AI needs to be examined, as at some point it seems they have simply stopped making any improvements.

We have not used Monte Carlo's AI capabilities significantly. We primarily use it for investigating alerts from time to time. However, we do not use it extensively, so I do not think it is fair to comment comprehensively on it.

Their incident tracking and incident debugging bot is useful for new analysts who are starting onboard. It helps them debug incidents, get a clearer picture, and achieve a clear head start to reach the root of the problem faster. Regarding accuracy and reliability, I would rate it at eighty to eighty-five percent. Given the current inherent non-reliability of AI models, every single thing that Monte Carlo says needs to be validated.

I have been using Monte Carlo for the last three years.

Monte Carlo is a fairly stable product.

Monte Carlo is robust and scalable for our data needs. We have not encountered any issues or challenges with the scalable platform.

We did not previously use any other solutions.

Time has been saved in reporting errors, SLAs, and performing reloads because we have been able to catch data errors faster. We estimate approximately eight to ten percent time savings, but regarding money savings and fewer employees needed, I do not think we can achieve that.

We have seen a reduction of incidents in approximately seven to eight percent in production scenarios, which has definitely been positive. I recommend checking out Monte Carlo to see if it fits your data-related needs. Conduct a thorough proof of concept, review the licensing and contract agreements, and if it meets your requirements, proceed with it. I would rate this review at seven out of ten.

My main use case for Monte Carlo is to create monitors to monitor data quality. I created different types of monitors, metrics, SQL, and custom SQL. I also utilize the Monte Carlo API to ingest data and create a datamart for analytical purposes.

Recently, I used the Monte Carlo API to ingest data to create a datamart, where I created several tables, such as the monitor snapshot, alerts, the run monitor history, and the monitor credit consumption.

Other use cases I have for Monte Carlo include creating a monitor, where I can write custom SQL to monitor data quality. If the data volume fluctuates dramatically, either increasing or decreasing too much, it will fire an alert.

Monte Carlo offers several great features that stand out to me. Monte Carlo's use of machine learning and AI to determine the threshold for the monitors that trigger alerts is particularly impressive.

Recently, Monte Carlo added features based on AI that respond to alerts. When you see an alert, the AI can help you analyze it and find the root cause.

Monte Carlo has positively impacted my organization by ensuring the data quality of the data pipeline, which is essential for data-driven decision-making. The company's business depends on data-driven decision-making, so data quality is very important. Monte Carlo monitors data quality issues and helps identify and fix those issues efficiently.

I estimate that I resolve data issues roughly 30% more efficiently since using Monte Carlo.

Monte Carlo adopted AI just recently, so there is room for improvement in the accuracy of the AI.

I have been using Monte Carlo for two years.

Monte Carlo is stable.

Monte Carlo's scalability is good. As our company's business grows and the data volume increases, Monte Carlo scales very well.

I find Monte Carlo's customer support to be good, and their customer support team responds very fast. We can submit a support ticket, and they also provide a platform using AI to answer questions. If the AI cannot resolve the issue, a ticket is created, and a team member will help.

I did not use a different solution before Monte Carlo.

My experience with pricing, setup costs, and licensing is limited as that falls under the management team's responsibility.

I am not responsible for how Monte Carlo is deployed in my organization. I believe the platform team is responsible for that.

I do not have a specific number regarding the return on investment with Monte Carlo, but I believe it saves a lot of workload for employees if we do not use it.

My experience with pricing, setup costs, and licensing is limited as that falls under the management team's responsibility.

I did not evaluate other options before choosing Monte Carlo. When I joined the company, we were already using it.

Monte Carlo does a good job overall. I rate Monte Carlo an 8 out of 10.

I chose eight because Monte Carlo does a good job, even though there are places for improvement. Overall, it is good.

Regarding Monte Carlo's AI capabilities, I find its accuracy and reliability of output to be good. Even though it is not 100% accurate, it is still good. A human review is necessary, but it does a good job.

My advice to others looking into using Monte Carlo is that it is a good product.

Overall, I think Monte Carlo is good, and the questions cover almost all the aspects. My overall review rating for Monte Carlo is 8.

My use case for Monte Carlo is both data quality and observability. We are using it as part of robustly monitoring the jobs as well as finding out any anomalies with respect to data quality issues.

In Monte Carlo, as part of observability, we have dynamic alert systems that learn the previous patterns of data anomalies and customize the monitoring system. It does not only have static rules because it has machine learning based models that learn the patterns. For example, during Thanksgiving, more purchases are happening, so you can expect more issues. It learns those patterns and sends the alerts based on that. The system does not send false alerts.

We use anomaly detection as part of a monitoring system. For instance, I was working for an airline where daily check-ins, checkouts, and transactions happen in real time. We wanted a very robust monitoring system that could monitor the data in real time. Whenever there is an anomaly, such as some columns which are not supposed to have nulls or which are not supposed to have certain data, you can train your machine learning model to have that threshold. You cannot just keep that threshold at 10% or something. You can train that machine learning model so that whenever a null detection happens or some kind of data mismatch happens, or when there is a schema change, it detects so many anomalies. We had many anomaly detection alerts.

The customizable alerts and dashboards in Monte Carlo were very customizable because it not only gives you the option to select alert features using drop downs, it also opens up a window where you can write your own customizable queries in SQL.

Certain features of Monte Carlo have contributed to maintaining our data trust by having multiple steps where you can define the model and also specify the probability distribution for your input. You can simulate that model over the past pattern. Additionally, it will give nice dashboards which are very handy and easy to understand to check how the anomaly patterns are progressing. If there is any sudden spike on a particular day, you can easily spot that and dig deep into it.

Regarding Monte Carlo, I would say that currently we can have machine learning options. We might have to integrate MCP servers so that it can connect to multiple systems at once and we should have some kind of a placeholder for artificial intelligence integration. Artificial intelligence can access multiple systems underneath Monte Carlo, such as any kind of database or any kind of real-time source systems. Currently, I think it is lacking that capability.

I have been using the solution since 2024, which is around more than two years.

The stability was very stable. I did not see any issues with respect to stability, and I would rate it a ten.

The scalability was good because when we enrolled it, it was already scaled up. We did not require it to be scaled up again, so I cannot fully comment on that.

I rate the technical support around nine out of ten because they are pretty responsive.

Data quality monitoring throughout the data lifecycle is very important, especially in this artificial intelligence era. If you feed garbage into artificial intelligence, it will hallucinate more and will not give you accurate results. It might divert into deploying many more agents and utilizing many more tokens rather than confining to a particular set of tokens. It is not only important from your data perspective, but also very important from your revenue perspective. The lost tokens are directly impacting an increase in costs or a decrease in revenue.

Regarding the pricing, it is a bit expensive compared to traditional monitoring systems provided by other vendors. However, the extra features and the trust come with some cost, so I think it should be fine. I have worked with many customers who do not have any complaints. In fact, they migrated many other systems from traditional monitoring systems to Monte Carlo. The customers are accepting of this pricing model.

Monte Carlo has many advantages compared to other solutions. As I mentioned, it has a lot of machine learning functionality and excellent user friendliness. The interface is quite crisp and the appearance is quite good. Traditional tools require some prior knowledge, but with Monte Carlo, you can onboard any user at any time. They can easily understand how to use that tool.

The solution requires maintenance because new features get rolled out and you need to upgrade those features. During that time there is a little bit of a pain point, but that is acceptable because you will experience new functionality.

If others are looking to implement this product, my advice is to robustly monitor their system with very little human intervention. Monte Carlo has an option where it will directly allow you to dig deep into the root cause and you just need to do a few clicks and it will get you to that data issue where it is happening. Very little human intervention is required for this. I give this solution an overall rating of eight out of ten.