Amazon SageMaker DeepAR now supports missing values, categorical and time series features, and generalized frequencies

Today we are launching several new features for DeepAR in Amazon SageMaker. DeepAR is a supervised machine learning algorithm for time series prediction, or forecasting, that uses recurrent neural networks (RNNs) to produce probabilistic forecasts. Since its launch, the algorithm has been used for a variety of use cases. We are excited to give developers access to new features: support for missing values, user-provided time series features, multiple categorical features, and generalized frequencies.

Forecasting is everywhere

Forecasting can improve business processes across many industries. This makes forecasting an ideal entry point into the world of automation and optimization using machine learning (ML) and Artificial Intelligence (AI). Whether you optimize the supply chain through better product demand forecasts, allocate computing resources more effectively by predicting web server traffic, or save lives by staffing hospitals to meet patient needs, there are few domains where forecasting does not quickly return its investment.

At Amazon, we use forecasting to drive automated business decision making in a variety of domains. Some of these include forecasting the product and labor demand in our fulfillment centers or forecasting capacity for AWS services.

DeepAR algorithm new features

In this blog post we’ll give you a quick overview of the new features of the DeepAR algorithm that are now available.

Missing value support

Although forecasting algorithms (or their implementations) often assume that data is complete, reality tends to disagree. In practice, time series data is often only partially observed. For example, a time series may contain sensor data, and that sensor might be out of order or offline for a given time; a product might be out of stock when predicting demand, or the traffic time series information of a server might be unavailable for several days due to downtime or misconfiguration. In such cases, the burden often lies on the user who has to fill (impute) the missing values before using the forecasting algorithm. However, incorrectly imputed data can have detrimental effects on the forecast. For example, if you replace unobserved values by zeros in out-of-stock situations, the forecasts will be biased down toward zero. Surely, this is unwanted if you want to predict future product demand, especially after long out-of-stock phases.

The latest release of DeepAR now handles missing values directly within the model. This makes forecasting with time series that contain missing values easier (no imputation during pre-processing is required) and more accurate (by utilizing the RNN model instead of relying on coarse-grained external imputation techniques).

Here is an example of how the model can generate forecasts for time series with missing values:

Custom time series feature

One of the top customer requests for DeepAR has been the ability to supply custom time-varying features to the model. When forecasting a time series, users often have other, explanatory time series that can provide useful information about the target time series. For instance, when predicting demand for a product, you might have access to time-dependent information, such as whether a promotion has been in effect during a certain period of time or what the price of the product was. You can now provide your own feature time series to model custom seasonality patterns, causal effects, and more (in addition to the default seasonality features that DeepAR provides internally). Using this time-varying information can ultimately improve accuracy by “explaining” effects that otherwise would have to be attributed to noise. In addition, you can simulate “what if” scenarios, for example, “What happens if I promote this item in the future for one week?”.

Here is an example that uses a time-varying feature (the blue line) that correlates with increased short-term demand. In practice, this could be a promotion or a special calendar event. In this example, the feature allows DeepAR to distinguish periods with increased demand from the other periods.

Multiple categories per time series

When forecasting a given time series, group information can be very helpful, especially when there is not much historical data available. DeepAR previously supported grouping time series by a single categorical grouping attribute, allowing users to encode one possible grouping of the time series. In practice, time series are often associated with not just one but multiple groupings that can be leveraged for extracting fine-grained group-level effects, such as seasonality patterns. For example, when forecasting the demand for a product, the product might be part of multiple groups: It might be associated with a high-level group such as “clothing” (instead of “electronics”), or lower-level product groups such as “shoes” (instead of “t-shirts”) or and “flip-flops” (instead of “sneakers”). Or it might be a group that is associated with the product brand. Or it could be a group associated with store the product is sold at, and so on. Support for multiple groupings was one of the features customers requested most, and the new release of DeepAR now supports this. The grouping information can now be encoded into multiple categorical features, one per grouping, and supplied to DeepAR as a vector of integers per time series. By making use of this grouping information, DeepAR can learn group-specific behavior such as seasonality patterns, ultimately increasing forecast accuracy.

Generalized frequencies

Up to now DeepAR supported a fixed set of time frequencies that are most commonly found in time series data (minute, hour, day, week, or month). Of course, time series data can be sampled at resolutions other than these basic frequencies. For instance, a sensor could acquire data points every 15 minutes, or product sales might be only recorded biweekly. The new release now supports integer multiples of any of the base frequencies, making it possible to handle such data without further pre-processing.

Notebook on a real-world dataset

Finally, we are releasing a notebook that shows you how to process a real-world dataset on Amazon SageMaker with DeepAR. This dataset consists of the hourly electricity consumption of 370 customers. It has been used in several academic papers, including this DeepAR paper. The notebook shows you how to use the new features of this release on a real use case.

Our notebook also demonstrates how to call a model deployed in Amazon SageMaker to generate forecasts on the fly.

Learn more

For more information, see the DeepAR documentation. For more details on the mathematics behind DeepAR, see the academic paper written by several Amazon machine learning scientists.

Additional reading

We recently published an introductory article on neural forecasting in the summer 2018 issue of Foresight called “Deep Learning for Forecasting.” Authors are Tim Januschowski, Jan Gasthaus, Yuyang Wang, Syama Rangapuram, and Laurent Callot.

About the Authors

Valentin Flunkert is a Senior Machine Learning Scientist at AWS AI Labs. He is passionate about building machine learning systems for solving business problems. He has worked on a variety of machine learning and forecasting problems at Amazon. 

Syama Sundar Rangapuram is a Machine Learning Scientist at AWS AI Labs. His research interests are in machine learning and optimization. In forecasting, he has worked on probabilistic models and data-driven models in particular for the cold-start problem.

Alexander Alexandrov is a Post-Doc on the AWS AI Labs team and TU-Berlin. He is passionate about scalable data management, data analytics applications, and optimizing DSLs. 

Jan Gasthaus is a Senior Machine Learning Scientist with AWS AI Labs where his passion is designing machine learning models, algorithms, and systems, and deploying them at scale.

Michael Bohlke-Schneider is a Data Scientist in AWS AI Labs/Fulfillment Technology, researching and developing forecasting algorithms in SageMaker and applying forecasting to business problems.

David Salinas is a Senior Applied Scientist in the AWS AI Labs team. He is working on applying deep-learning to various application such as forecasting or NLP.

Lorenzo Stella is an Applied Scientist on the AWS AI Labs team. His research interests are in machine learning and optimization. He has worked on probabilistic and deep models for forecasting.

Tim Januschowski is a Machine Learning Science Manager at AWS AI Labs. He has worked on forecasting and has produced end-to-end solutions for a wide variety of forecasting problems, from demand forecasting to server capacity forecasting over the course of his tenure at Amazon.