CSSS Seminar: Empirical Bayes for A/B testing and meta-analysis: a spectral approach (5/6/2020)
Posted: 5/5/2020 (Local Events)
Wednesday, May 6th, 2020 – 12:30 pm
Large-scale A/B testing is increasingly prevalent in many industries. We propose an empirical Bayes approach, which assumes that the treatment effects are realized from a “true prior”. This requires inferring the prior from previous experiments. Following Robbins, we estimate a family of marginal densities of empirical effects, indexed by the noise scale. We show that this family is characterized by the heat equation. We develop a spectral MLE based on Fourier series, which can be efficiently computed via convex optimization. We select hyperparameters and compare models using two model selection criteria. Our method is demonstrated on experimentation data from Amazon.com. The same method can also be applied to meta-analysis.
Note: This is joint work with James McQueen (Amazon.com) and Thomas Richardson (UW and Amazon.com).
Preprint: http://arxiv.org/abs/2002.02564
Date: 05/06/2020
Time: 12:30 pm