Can We Trust Shoestring Evaluations?

Ravallion, Martin

关键词: AGRICULTURE;    ANTIPOVERTY PROGRAM;    CLINICS;    CORRELATIONS;    DATA COLLECTION;   

DOI  :  10.1596/1813-9450-5983
RP-ID  :  WPS5983

学科分类：社会科学、人文和艺术（综合）

来源: World Bank Open Knowledge Repository

【 摘 要 】

Many more impact evaluations could bedone, and at lower unit cost, if evaluators could avoid theneed for baseline data using objective socio-economicsurveys and rely instead on retrospective subjectivequestions on how outcomes have changed, askedpost-intervention. But would the results be reliable? Thispaper tests a rapid-appraisal, "shoestring,"method using subjective recall for welfare changes. Therecall data were collected at the end of a full-scaleevaluation of a large poor-area development program inChina. Qualitative recalls of how living standards havechanged are found to provide only weak and biased signals ofthe changes in consumption as measured from contemporaneoussurveys. Importantly, the shoestring method was unable tocorrect for the selective placement of the program favoringpoor villages. The results of this case study are notencouraging for future applications of the shoestringmethod, although similar tests are needed in other settings.