Posted On: May 10, 2023
Amazon SageMaker Autopilot, a low-code machine learning (ML) service which automatically builds, trains and tunes the best ML models, now supports training with weighted objective metrics in Ensemble mode and also supports eight additional objective metrics. Assigning weights to each data sample in the training data set can improve overall model performance by helping the model learn better, reduce bias towards a particular class, and increase stability.
When training upon imbalanced datasets where some classes have significantly fewer data samples than others, assigning higher weights to those can help the model learn better and reduce bias towards the majority classes. Starting today, you can pass a weight column name in your input dataset while creating an Autopilot experiment. SageMaker Autopilot will use these weight values to learn more about your dataset and apply the learnings while training the ML model.
SageMaker Autopilot now also supports eight additional objective metrics such as RMSE, MAE, R2, Balanced Accuracy, Precision, Precision Macro, Recall and Recall Macro (documented here). The selected objective metric is optimized during training to provide the best estimate for model parameter values from the data. If you do not specify a metric explicitly, the default behavior is to automatically use MSE for regression, F1 for binary classification and Accuracy for multi-class classification.
To get started, Create an SageMaker Autopilot experiment in SageMaker Studio console. Upgrade to the latest version of SageMaker Studio to use the new sample weights column feature and additional set of objective metrics. To learn more, refer to the developer guide and product page.