Reviews from AWS Marketplace
0 AWS reviews
-
5 star0
-
4 star0
-
3 star0
-
2 star0
-
1 star0
External reviews
External reviews are not included in the AWS star rating for the product.
Self-Serve Observability Platform
What do you like best about the product?
WhyLabs is the second observability platform I have ever used, and I can say the core features I like about the platform is that it is easy to set up and implement the features, the checks and metrics were already pre-loaded so I did not need to do much in configuring the application, and monitoring was not difficult to get started with. It also integrates well with the serving and data libraries we used for the production tutorial setup.
What do you dislike about the product?
Nothing so far, I only experienced a stability issues once (sometime in 2022), but support was able to help me quickly fix it.
What problems is the product solving and how is that benefiting you?
Since 2022, I have sparesely used WhyLabs to monitor the quality of datasets for one client and 2 customers (because it was not their core requirment but a nice piece of their stack to have).
whylogs seemed like the perfect choice for a consultant that clients did not want to entirely release their data to; I found that it only captures the profile and stats info instead of the raw data here.
Rcently, I started testing out LLM security features with LangKit and I cannot believe how quick it is to use. I followed a workshop few months ago that showed me how to detect jailbreak attempts and toxicity in LLM inputs and outputs using LangKit. Took that learning and now with a client's project, we have tested out logging the telemetary data from the evaluation to WhyLabs. Looks good so far, so once I upgrade the pricing limit for this client, we plan to scale our usage here. Excited about this one.
whylogs seemed like the perfect choice for a consultant that clients did not want to entirely release their data to; I found that it only captures the profile and stats info instead of the raw data here.
Rcently, I started testing out LLM security features with LangKit and I cannot believe how quick it is to use. I followed a workshop few months ago that showed me how to detect jailbreak attempts and toxicity in LLM inputs and outputs using LangKit. Took that learning and now with a client's project, we have tested out logging the telemetary data from the evaluation to WhyLabs. Looks good so far, so once I upgrade the pricing limit for this client, we plan to scale our usage here. Excited about this one.
- Leave a Comment |
- Mark review as helpful
Top notch features at an affordable price
What do you like best about the product?
I've used WhyLabs for a few weeks and I was extremely pleased with it!
I will evaluate some dimensions of the tool that summarize my experience with it.
Easy Data Ingestion:
The ingestion API is straightforward to use and supports multiple connectors such as BigQuery, Databricks, and Spark, making data importation easy. Whylabs' use of Data Profiling ensures fast and secure data processing, eliminating the need to upload entire datasets, and making all the process very secure, since your data doesn't leave your servers.
Reliable Data Features:
Whylabs delivers all standard feature metrics accurately. Tracking data and model drift is very straightforward using Monitors.
Also, the platform supports custom metrics creation during or after ingestion.
Grouping by variables (segments) works well but must be defined during ingestion. Then you can analyze dataset features and track model performance per segment.
Flexible Monitors:
The monitoring system in Whylabs is highly adaptable and user-friendly, covering multiple variables with ease.
Monitors are easy to set up via the UI or JSON import, with summarized notifications for each monitor, keeping users informed without overwhelming them.
Additionally, monitors are JSON serializable, which is very helpful since you can track them with version control.
User-Friendly Usability:
Whylabs have a clean and intuitive UI, simplifying navigation for users.
While some advanced features may require programming knowledge, most tasks can be accomplished within the UI.
Thanks to data profiling, Whylabs delivers speedy performance without compromising on accuracy.
Solid Documentation:
The documentation provided by Whylabs is comprehensive and easy to understand, enabling users to make the most of the platform.
Pricing:
It's simply cheaper than its competition while having top notch features.
Customer Support:
They are always very helpful, answering all our questions and having several calls showcasing us different uses cases directly on the platform.
Overall, Whylabs offers a straightforward, efficient and affordable solution for monitoring Machine Learning models, with easy data ingestion, reliable feature analysis, and flexible monitoring options.
I will evaluate some dimensions of the tool that summarize my experience with it.
Easy Data Ingestion:
The ingestion API is straightforward to use and supports multiple connectors such as BigQuery, Databricks, and Spark, making data importation easy. Whylabs' use of Data Profiling ensures fast and secure data processing, eliminating the need to upload entire datasets, and making all the process very secure, since your data doesn't leave your servers.
Reliable Data Features:
Whylabs delivers all standard feature metrics accurately. Tracking data and model drift is very straightforward using Monitors.
Also, the platform supports custom metrics creation during or after ingestion.
Grouping by variables (segments) works well but must be defined during ingestion. Then you can analyze dataset features and track model performance per segment.
Flexible Monitors:
The monitoring system in Whylabs is highly adaptable and user-friendly, covering multiple variables with ease.
Monitors are easy to set up via the UI or JSON import, with summarized notifications for each monitor, keeping users informed without overwhelming them.
Additionally, monitors are JSON serializable, which is very helpful since you can track them with version control.
User-Friendly Usability:
Whylabs have a clean and intuitive UI, simplifying navigation for users.
While some advanced features may require programming knowledge, most tasks can be accomplished within the UI.
Thanks to data profiling, Whylabs delivers speedy performance without compromising on accuracy.
Solid Documentation:
The documentation provided by Whylabs is comprehensive and easy to understand, enabling users to make the most of the platform.
Pricing:
It's simply cheaper than its competition while having top notch features.
Customer Support:
They are always very helpful, answering all our questions and having several calls showcasing us different uses cases directly on the platform.
Overall, Whylabs offers a straightforward, efficient and affordable solution for monitoring Machine Learning models, with easy data ingestion, reliable feature analysis, and flexible monitoring options.
What do you dislike about the product?
There are a few cons:
- Dashboards are in beta, and while functional, they lack polish in terms of user interface. They are working actively on this, so probably a few months after this review this may be already fixed.
- Defining groupings by variables must be done at ingestion time, limiting flexibility for post-ingestion analysis.
That being said, they are very open to feedback and they may change or add features based on your needs.
In our case, dashboards were important and they are working on them.
- Dashboards are in beta, and while functional, they lack polish in terms of user interface. They are working actively on this, so probably a few months after this review this may be already fixed.
- Defining groupings by variables must be done at ingestion time, limiting flexibility for post-ingestion analysis.
That being said, they are very open to feedback and they may change or add features based on your needs.
In our case, dashboards were important and they are working on them.
What problems is the product solving and how is that benefiting you?
It solves most of the ML model monitoring needs that ML models often have while being affordable.
Simplifying Complexity in LLM Monitoring
What do you like best about the product?
WhyLabs is an exceptional observability tool for applications built with large language models. Its ease of use enables us to integrate langkit into the existing architecture. Additionally, they have an excellent customer support community.
What do you dislike about the product?
Building custom applications with large language models is a challenging experience. Better documentation about how to effectively use Whylabs in monitoring applications with LLMs would be helpful.
What problems is the product solving and how is that benefiting you?
WhyLabs is very helpful in monitoring applications built with large language models by providing a highly simplified observability tool. This has overcome the challenges we were facing with our productionized application.
Monitoring LLMs for succees!
What do you like best about the product?
The team behind WhyLabs is awesome. I like how easy it is to get started with their platform, their commitment to an open-source approach, and their active engagement with the AI community with regular workshops and education around cutting-edge monitoring and evaluation techniques.
What do you dislike about the product?
Having more flexibility in visualizations and easier ways to share them outside the product would be nice.
What problems is the product solving and how is that benefiting you?
I use WhyLabs to help keep a pulse on LLMs by monitoring valuable metrics such as jailbreak scores, sentiment, toxicity, and readability. This has helped me catch problems early and gives me some good metrics to compare when finetuning models or adjusting prompts.
WhyLabs Platform - offering intuitive insights into AI model behavior.
What do you like best about the product?
My experience with the WhyLabs Platform was enlightening, offering intuitive insights into AI model behavior and enhancing my understanding of AI observability. The best part of about the interface is its user friendly interface and comprehensive suite of analytics which provide valuable insights into AI model behavior and which would also enable a dashboard for data monitoring, model features. It also facilitates real time monitoring, ultimately enhancing efficiency of AI development processes. It also gives you the leverage to customise the model features as per the project requirement.
What do you dislike about the product?
Few advanced customisation options for analytics is not available as of now, but as informed it would be available soon as an update in the platform. So that solves the downside down the line.
What problems is the product solving and how is that benefiting you?
The main things which is being helped with is the data observability and monitoring. It could help me to effectively integrate and monitor the behavior of AI models, enabling to detect issues like data drift, model degradation and biases. This in turn helps in mitigating risks and optimizing model performance.
Easy Integration and Outstanding LLM Monitoring with WhyLabs AI Observatory
What do you like best about the product?
The foremost aspect that stood out for us was the ease of integration. It was quite straightforward, almost as same as the sample code to put LangKit into our LLM pipeline and upload it to WhyLabs. The getting-started docs were clear and concise. Even for companies with limited resources like ours, WhyLabs AI Observatory proves to be an accessible and easily adaptable solution.
What do you dislike about the product?
The data visualization tools are highly capable, although there is some learning curve to master it. However, their customer success and support team are responsive, helpful, and have consistently gone above and beyond to ensure that we utilize the platform to its full potential.
What problems is the product solving and how is that benefiting you?
The monitoring capabilities of WhyLabs AI Observatory are the main reason why we use it. We have very limited control of external LLM services. WhyLabs provides alerts and insights into those LLM models, enabling us to identify and address potential issues promptly.
Super powerful and simple
What do you like best about the product?
WhyLabs is so useful for debugging, maintaining, and developing production ML systems. It's portable, highly configurable, and provides unified visibility across a broad range of systems/services in a production ML ecosystem. It dramatically improves ML devops and is incredibly easy to get started. It integrates very easily with all the systems I've used, implementation is a breeze, and there's very low friction to get started. I've caught tons of issues using it.
What do you dislike about the product?
I don't really have any complaints about it, it does its thing incredibly well for my use cases.
What problems is the product solving and how is that benefiting you?
Visibility and observability that is unified across all stages of ML dev and ops: model training, development, deployment, testing, etc. Importantly it integrates with a broad range of systems/languages.
It accelerates development/deployment by provide great visibility.
For ops, it has caught so many super costly issues which everything else misses, especially things related to train/serve skew or semantic drift.
Seriously, it's a must-have for productionizing ML.
It accelerates development/deployment by provide great visibility.
For ops, it has caught so many super costly issues which everything else misses, especially things related to train/serve skew or semantic drift.
Seriously, it's a must-have for productionizing ML.
WhyLabs answers Why
What do you like best about the product?
WhyLabs Al is easy to understand and use. It enhances interpretability, making it easier for data scientists and machine learning engineers to understand model behavior
What do you dislike about the product?
More detailed dashboard would be more helpful
What problems is the product solving and how is that benefiting you?
It enable me monitoring and understanding the interaction between my data and model
Intuitive and user-friendly product.
What do you like best about the product?
I see so much potential how it can be applied to my own academic work. It's very friendly by providing easy-to-read plots and intuitive platform.
What do you dislike about the product?
I found some of the features very slow to display.
What problems is the product solving and how is that benefiting you?
I don't have any specific problem at the moment, but I see potential in using it to conduct a short text analysis as an exploratory method.
Amazing 1:1 Technical Support
What do you like best about the product?
While we have not gone extremely deep into all the features that Whylabs provides, we have had extraordinary 1:1 technical support. They are quick to respond, fast at finding solutions, and eager to make improvements for your use case.
What do you dislike about the product?
There is some complexity setting up data that lags behind the current date but with some changes you can easily overcome this.
What problems is the product solving and how is that benefiting you?
Whylabs helps us monitor our models in production.
showing 1 - 10