Andrew Gelman.
The Centre for Central Banking Studies recently hosted their annual Chief Economists Workshop, whose theme was “What can policymakers learn from other disciplines”. In this guest post, one of the keynote speakers at the event, Andrew Gelman professor of statistics and political science at Columbia University, points out some of the pitfalls of randomly assigned experiments with control groups.
When studying the effects of interventions on individual behavior, the experimental research template is typically: Gather a bunch of people who are willing to participate in an experiment, randomly divide them into two groups, assign one treatment to group A and the other to group B, then measure the outcomes. If you want to increase precision, do a pre-test measurement on everyone and use that as a control variable in your regression. But in this post I argue for an alternative approach- study individual subjects using repeated measures of performance, with each one serving as their own control.