Regression Adjustment with Synthetic Controls in Online Experiments

Conference on Digital Experimentation at MIT (CODE)


In the setting of online experiments, we propose a two-step procedure to improve efficiency for estimating average treatment effect (ATE) by combining synthetic control methods with the popular regression adjustment framework. In particular, we form a synthetic control for each and every subject in the experiment using a donor pool that consists of k nearest-neighbors (kNN) from outside of the experiment. The predicted counterfactuals are then used in the following regression adjustment step. The asymptotic theory of the method can be shown following and is validated in a realistically calibrated Monte Carlo analysis. For both user-level and cluster experiments at Facebook, we show that the proposed method yields significantly narrower CIs compared with the standard difference-in-mean estimator and a widely used OLS adjusted estimator.

Related Publications

All Publications

NeurIPS - December 6, 2021

Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement

Samuel Daulton, Maximilian Balandat, Eytan Bakshy

KAIS - June 30, 2021

Fire Now, Fire Later: Alarm-Based Systems for Prescriptive Process Monitoring

Stephan A. Fahrenkrog-Petersen, Niek Tax, Irene Teinemaa, Marlon Dumas, Massimiliano de Leoni, Fabrizio Maria Maggi, Matthias Weidlich

The Quarterly Journal of Economics - October 25, 2021

Peer Effects in Product Adoption

Michael Bailey, Drew Johnston, Theresa Kuchler, Johannes Stroebel, Arlene Wong

IEEE Sensors Journal - July 19, 2021

OpenEDS2020 Challenge on Gaze Tracking for VR: Dataset and Results

Cristina Palmero, Abhishek Sharma, Karsten Behrendt, Kapil Krishnakumar, Oleg V. Komogortsev, Sachin S. Talathi

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookie Policy