Beyond Baseline and Follow-up : The Case for More T in Experiments

McKenzie, David

关键词: AUTOCORRELATION;    BOOTSTRAP;    CHOLESTEROL;    CLINICAL TRIALS;    CONFIDENCE INTERVALS;   

DOI  :  10.1596/1813-9450-5639
RP-ID  :  WPS5639

学科分类：社会科学、人文和艺术（综合）

来源: World Bank Open Knowledge Repository

【 摘 要 】

The vast majority of randomizedexperiments in economics rely on a single baseline andsingle follow-up survey. If multiple follow-ups areconducted, the reason is typically to examine the trajectoryof impact effects, so that in effect only one follow-upround is being used to estimate each treatment effect ofinterest. While such a design is suitable for study ofhighly autocorrelated and relatively precisely measuredoutcomes in the health and education domains, this papermakes the case that it is unlikely to be optimal formeasuring noisy and relatively less autocorrelated outcomessuch as business profits, household incomes andexpenditures, and episodic health outcomes. Taking multiplemeasurements of such outcomes at relatively short intervalsallows the researcher to average out noise, increasingpower. When the outcomes have low autocorrelation, it canmake sense to do no baseline at all. Moreover, the authorshows how for such outcomes, more power can be achieved withmultiple follow-ups than allocating the same total samplesize over a single follow-up and baseline. The analysishighlights the large gains in power from ANCOVA rather thandifference-in-differences when autocorrelations are low anda baseline is taken. The paper discusses the issues involvedin multiple measurements, and makes recommendations for thedesign of experiments and related non-experimental impact evaluations.