Testing the Retrospective Pretest with High School Youth in Out-of-School Time Programs

Journal of Youth Development

Jill Young¹  Leanne Kallemeyn² 

[1] After School MattersLoyola University Chicago;Loyola University Chicago

关键词: out-of-school time;    high school;    program evaluation;    survey;    pretest;    response-shift bias;   

DOI  :  10.5195/jyd.2019.635

学科分类：社会科学、人文和艺术（综合）

来源: University Library System, University of Pittsburgh

【 摘 要 】

Practitioners and evaluators face several constraints in conducting rigorous evaluations to determine program effect. Researchers have offered the retrospective pretest/posttest design as a remedy to curb response-shift bias and better estimate program effects. This article presents an example of how After School Matters (ASM) tested the use of retrospective pretest/posttest design for evaluating out-of-school time (OST) programs for high school youth participants. Differences between traditional pretest and retrospective pretest scores were statistically significant, but effect sizes were negligible, indicating that both pretests yielded similar results. Interviews with youth led to 3 key findings that have implications for ASM using retrospective pretests with youth: response-shift bias was more prominent in youth interviews than in quantitative findings, youth recommended reordering the questions so that the retrospective pretest appears first to increase comprehension, and acquiescence bias emerged in the interviews. This study demonstrates that the retrospective pretest/posttest design can be an alternative to the traditional pretest/posttest design for OST at ASM. These findings are important for ASM and other youth-serving organizations, which often have limited capacity to survey youth multiple times within 1 program session.

【 授权许可】

CC BY