A/B Testing at Scale – Amazon Machine Learning Research

This week, Amazon presented an academic paper at KDD 2017, the prestigious machine learning and big data conference. The paper shows Amazon’s research into tools that help us measure customers’ satisfaction and better learn how we can implement ideas that delight them. Specifically, we show an efficient bandit algorithm for multivariate testing, where one seeks to find an optimal series of actions with as little experimental effort as possible. One application of this research, for example, is optimizing the layout of a web page.

Please check out this fun, three-minute video that explains the paper and how the ideas are applied within Amazon. Also, it won the KDD 2017 Audience Appreciation Award!

Download the paper from KDD.org: An efficient bandit algorithm for realtime multivariate optimization.