Sign in Agent Mode
Categories
Your Saved List Become a Channel Partner Sell in AWS Marketplace Amazon Web Services Home Help

Easy to use, but super helpful tool for logging machine learning experiments

  • By Manufacturing
  • on 06/13/2024

What do you like best about the product?
That it is super easy to log all metrics, loss curves and all different kind of data and to get that data visualized in an interpretable manner. I really like how it is integrated into other frameworks like eg pytorch lightning.
I use W&B almost daily, at least I haven't started a single training run without using W&B ever since I subscribed.
What do you dislike about the product?
1) I would like to customize the plots and visualizations of the metrics shown even more. That would be nice to be able to do by using a script or something that can be used across multiple projects --> eg I would like to determine (from my python script) the color of all lines in a plot, and I don't know how to do that.
It would just be usefull for communication to team leaders, bosses etc., if they knew that eg. accuracy would also be plotted with a red, dotted line and that recall would always be a thick, blue line or something like that.

2) I have a hard time figuring out how to navigate all artifacts and how to use those. In eg the integration with ultralytics W&B will create an artifact for each epoch, which quickly fills up my storage. However, that is just a minor thing as I have just created a stand alone script to delete artifacts that aren't tagged with "best" etc.
What problems is the product solving and how is that benefiting you?
Before using W&B I would either use TensorBoard (hard to set up and won't log the same amount of data) or rely on the automatic logging from AWS SageMaker (which is crap). Both these older methods took really long time to setup which delayed all projects - and then we are not even talking about how hard it is to manage and remember "these files are from this run, these other files are from that other run" as that what needed before. Hence using W&B increases the frequency of which I can test new ideas by saving me so many hours each time I start on something new.