Comparison of six statistical methods for interrupted time series studies: empirical evaluation of 190 published series

BMC Medical Research Methodology

Monica Taljaard¹  Jeremy M. Grimshaw²  Simon L. Turner³  Andrew B. Forbes³  Joanne E. McKenzie³  Amalia Karahalios³ 

[1] Clinical Epidemiology Program, Ottawa Hospital Research Institute, 1053 Carling Ave, Ottawa, ON, Canada;School of Epidemiology and Public Health, University of Ottawa, 600 Peter Morand Crescent, K1G 5Z3, Ottawa, ON, Canada;Clinical Epidemiology Program, Ottawa Hospital Research Institute, 1053 Carling Ave, Ottawa, ON, Canada;School of Epidemiology and Public Health, University of Ottawa, 600 Peter Morand Crescent, K1G 5Z3, Ottawa, ON, Canada;Department of Medicine, University of Ottawa, Roger Guindon Hall, 451 Smyth Rd, Ottawa, ON, Canada;School of Public Health and Preventive Medicine, Monash University, Level 4, 553 St. Kilda Road, 3004, Melbourne, VIC, Australia;

关键词: Autocorrelation;    Interrupted Time Series;    Public Health;    Segmented Regression;    Statistical Methods;    Empirical study;   

DOI  :  10.1186/s12874-021-01306-w

来源: Springer

【 摘 要 】

BackgroundThe Interrupted Time Series (ITS) is a quasi-experimental design commonly used in public health to evaluate the impact of interventions or exposures. Multiple statistical methods are available to analyse data from ITS studies, but no empirical investigation has examined how the different methods compare when applied to real-world datasets.MethodsA random sample of 200 ITS studies identified in a previous methods review were included. Time series data from each of these studies was sought. Each dataset was re-analysed using six statistical methods. Point and confidence interval estimates for level and slope changes, standard errors, p-values and estimates of autocorrelation were compared between methods.ResultsFrom the 200 ITS studies, including 230 time series, 190 datasets were obtained. We found that the choice of statistical method can importantly affect the level and slope change point estimates, their standard errors, width of confidence intervals and p-values. Statistical significance (categorised at the 5% level) often differed across the pairwise comparisons of methods, ranging from 4 to 25% disagreement. Estimates of autocorrelation differed depending on the method used and the length of the series.ConclusionsThe choice of statistical method in ITS studies can lead to substantially different conclusions about the impact of the interruption. Pre-specification of the statistical method is encouraged, and naive conclusions based on statistical significance should be avoided.

【 授权许可】

CC BY