Graph Cluster Randomization: Network Exposure to Multiple Universes

ACM Conference on Knowledge Discovery and Data Mining (KDD)


A/B testing is a standard approach for evaluating the effect of online experiments; the goal is to estimate the `average treatment effect’ of a new feature or condition by exposing a sample of the overall population to it. A drawback with A/B testing is that it is poorly suited for experiments involving social interference, when the treatment of individuals spills over to neighboring individuals along an underlying social network. In this work, we propose a novel methodology using graph clustering to analyze average treatment effects under social interference. To begin, we characterize graph-theoretic conditions under which individuals can be considered to be ‘network exposed’ to an experiment. We then show how graph cluster randomization admits an efficient exact algorithm to compute the probabilities for each vertex being network exposed under several of these exposure conditions. Using these probabilities as inverse weights, a Horvitz-Thompson estimator can then provide an effect estimate that is unbiased, provided that the exposure model has been properly specified.

Given an estimator that is unbiased, we focus on minimizing the variance. First, we develop simple sufficient conditions for the variance of the estimator to be asymptotically small in n, the size of the graph. However, for general randomization schemes, this variance can be lower bounded by an exponential function of the degrees of a graph. In contrast, we show that if a graph satisfies a restricted-growth condition on the growth rate of neighborhoods, then there exists a natural clustering algorithm, based on vertex neighborhoods, for which the variance of the estimator can be upper bounded by a linear function of the degrees. Thus we show that proper cluster randomization can lead to exponentially lower estimator variance when experimentally measuring average treatment effects under interference.

Related Publications

All Publications

Weights and Methodology Brief for the COVID-19 Symptom Survey by University of Maryland and Carnegie Mellon University, in Partnership with Facebook

Neta Barkay, Curtiss Cobb, Roee Eilat, Tal Galili, Daniel Haimovich, Sarah LaRocca, Katherine Morris, Tal Sarig

arXiv - October 9, 2020

Country Differences in Social Comparison on Social Media

Justin Cheng, Moira Burke, Bethany de Gant

CSCW - October 17, 2020

The Determinants of Social Connectedness in Europe

Michael Bailey, Drew Johnston, Theresa Kuchler, Dominic Russel, Bogdan State, Johannes Stroebel

SocInfo - September 22, 2020

To help personalize content, tailor and measure ads, and provide a safer experience, we use cookies. By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies. Learn more, including about available controls: Cookies Policy