Natural experiments for the evaluation of place-based public health interventions: a methodology scoping review

Frontiers in Public Health

Public Health

Kate E. Mason¹  Elizabeth McGill²  Chiara Rinaldi²  Matt Egan²  David Taylor-Robinson³  Heather Brown⁴  Benjamin Barr⁵  Mhairi Campbell⁶  Peter Craig⁶  Marcia Gibson⁶  Falko F. Sniehotta⁷  Morgan Beeson⁸  Amy Dillon⁹  Katrina d'Apice⁹  Matthew Hickman⁹  Patricia N. Albers⁹  Cheryl McQuire⁹  Maxwell Fuller⁹  Kate Tilling^1,10  Frank de Vocht^1,11  Russell Jago^1,12  Anthony A. Laverty^1,13  Christopher J. Millett^1,13 

[1] Department of Public Health Policy and Systems, University of Liverpool, Liverpool, United Kingdom;Centre for Health Policy, University of Melbourne, Parkville, VIC, Australia;Department of Public Health, Environments and Society, London School of Hygiene and Tropical Medicine, London, United Kingdom;Department of Public Health, Policy and Systems. University of Liverpool, Liverpool, United Kingdom;Health Research, Lancaster University, Lancaster, United Kingdom;Institute of Population Health, University of Liverpool, Liverpool, United Kingdom;MRC/CSO Social and Public Health Sciences Unit, School of Health and Wellbeing, University of Glasgow, Glasgow, United Kingdom;NIHR Policy Research Unit Behavioural Science, Newcastle University, Newcastle upon Tyne, United Kingdom;Department of Public Health, Social and Preventive Medicine, Medical Faculty Mannheim, Heidelberg University, Heidelberg, Germany;Newcastle University Business School, Newcastle University, Newcastle upon Tyne, United Kingdom;Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom;Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom;MRC Integrative Epidemiology Unit, University of Bristol, Bristol, United Kingdom;Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom;The National Institute for Health Research, Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol and Weston NHS Foundation Trust, Bristol, United Kingdom;Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom;The National Institute for Health Research, Applied Research Collaboration West (NIHR ARC West), University Hospitals Bristol and Weston NHS Foundation Trust, Bristol, United Kingdom;Centre for Exercise, Nutrition and Health Sciences, School for Policy Studies, University of Bristol, Bristol, United Kingdom;School of Public Health, Imperial College London, London, United Kingdom;

关键词: public health;    public health policy;    natural experiments;    quasi experiments;    evaluations;    place-based;   

DOI  :  10.3389/fpubh.2023.1192055

 received in 2023-03-22, accepted in 2023-06-08,  发布年份 2023

来源: Frontiers

【 摘 要 】

IntroductionPlace-based public health evaluations are increasingly making use of natural experiments. This scoping review aimed to provide an overview of the design and use of natural experiment evaluations (NEEs), and an assessment of the plausibility of the as-if randomization assumption.MethodsA systematic search of three bibliographic databases (Pubmed, Web of Science and Ovid-Medline) was conducted in January 2020 to capture publications that reported a natural experiment of a place-based public health intervention or outcome. For each, study design elements were extracted. An additional evaluation of as-if randomization was conducted by 12 of this paper's authors who evaluated the same set of 20 randomly selected studies and assessed ‘as-if ' randomization for each.Results366 NEE studies of place-based public health interventions were identified. The most commonly used NEE approach was a Difference-in-Differences study design (25%), followed by before-after studies (23%) and regression analysis studies. 42% of NEEs had likely or probable as-if randomization of exposure (the intervention), while for 25% this was implausible. An inter-rater agreement exercise indicated poor reliability of as-if randomization assignment. Only about half of NEEs reported some form of sensitivity or falsification analysis to support inferences.ConclusionNEEs are conducted using many different designs and statistical methods and encompass various definitions of a natural experiment, while it is questionable whether all evaluations reported as natural experiments should be considered as such. The likelihood of as-if randomization should be specifically reported, and primary analyses should be supported by sensitivity analyses and/or falsification tests. Transparent reporting of NEE designs and evaluation methods will contribute to the optimum use of place-based NEEs.

【 授权许可】

Unknown   
Copyright © 2023 Albers, Rinaldi, Brown, Mason, d'Apice, McGill, McQuire, Craig, Laverty, Beeson, Campbell, Egan, Gibson, Fuller, Dillon, Taylor-Robinson, Jago, Tilling, Barr, Sniehotta, Hickman, Millett and de Vocht.