Making clinical trials more relevant: improving and validating the PRECIS tool for matching trial design decisions to trial purpose

Trials

Shaun Treweek¹  Peter Donnan³  Frank Sullivan³  Merrick Zwarenstein²  Kirsty Loudon³ 

[1] Health Services Research Unit, University of Aberdeen, 3rd Floor, Health Sciences Building, Aberdeen, Scotland, UK;Department of Family Medicine Director, Centre for Studies in Family Medicine, Schulich School of Medicine & Dentistry Western University, 245-100 Collip Circle, Research Park, London, ON N6G 4X8, Canada;Division of Population Health Sciences, University of Dundee, Kirsty Semple Way, Dundee DD2 4BF, UK

关键词: Applicability;    Trial design;    Clinical trials;    Explanatory;    Pragmatic;   

Others  :  1094186
DOI  :  10.1186/1745-6215-14-115

 received in 2013-01-09, accepted in 2013-04-08,  发布年份 2013

【 摘 要 】

Background

If you want to know which of two or more healthcare interventions is most effective, the randomised controlled trial is the design of choice. Randomisation, however, does not itself promote the applicability of the results to situations other than the one in which the trial was done. A tool published in 2009, PRECIS (PRagmatic Explanatory Continuum Indicator Summaries) aimed to help trialists design trials that produced results matched to the aim of the trial, be that supporting clinical decision-making, or increasing knowledge of how an intervention works. Though generally positive, groups evaluating the tool have also found weaknesses, mainly that its inter-rater reliability is not clear, that it needs a scoring system and that some new domains might be needed. The aim of the study is to: Produce an improved and validated version of the PRECIS tool. Use this tool to compare the internal validity of, and effect estimates from, a set of explanatory and pragmatic trials matched by intervention.

Methods

The study has four phases. Phase 1 involves brainstorming and a two-round Delphi survey of authors who cited PRECIS. In Phase 2, the Delphi results will then be discussed and alternative versions of PRECIS-2 developed and user-tested by experienced trialists. Phase 3 will evaluate the validity and reliability of the most promising PRECIS-2 candidate using a sample of 15 to 20 trials rated by 15 international trialists. We will assess inter-rater reliability, and raters’ subjective global ratings of pragmatism compared to PRECIS-2 to assess convergent and face validity. Phase 4, to determine if pragmatic trials sacrifice internal validity in order to achieve applicability, will compare the internal validity and effect estimates of matched explanatory and pragmatic trials of the same intervention, condition and participants. Effect sizes for the trials will then be compared in a meta-regression. The Cochrane Risk of Bias scores will be compared with the PRECIS-2 scores of pragmatism.

Discussion

We have concrete suggestions for improving PRECIS and a growing list of enthusiastic individuals interested in contributing to this work. By early 2014 we expect to have a validated PRECIS-2.

【 授权许可】

   
2013 Loudon et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

【 参考文献 】

• [1]Mitka M: FDA advisory decision highlights some problems inherent in pragmatic trials. JAMA 2011, 306:1851-1852.
• [2]Chalkidou K, Hoy A, Littlejohns P: Making a decision to wait for more evidence: when the National Institute for Health and Clinical Excellence recommends a technology only in the context of research. J R Soc Med 2007, 100:453-460.
• [3]Chalkidou K, Tunis S, Whicher D, Fowler R, Zwarenstein M: The role for pragmatic randomized controlled trials (pRCTs) in comparative effectiveness research. Clin Trials 2012, 9:436-446.
• [4]Schwartz D, Lellouch J: Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis 1967, 20:637-648.
• [5]Coca SG, Krumholz HM, Garg AX, Parikh CR: Underrepresentation of renal disease in randomized controlled trials of cardiovascular disease. JAMA 2006, 296:1377-1384.
• [6]Lee PY, Alexander KP, Hammill BG, Pasquali SK, Peterson ED: Representation of elderly persons and women in published randomized trials of acute coronary syndromes. JAMA 2001, 286:708-713.
• [7]Rothwell PM: Treating Individuals 1 - External validity of randomised controlled trials: “To whom do the results of this trial apply?”. Lancet 2005, 365:82-93.
• [8]Treweek S, Zwarenstein M: Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials 2009, 10:37. BioMed Central Full Text
• [9]Tunis SR, Stryer DB, Clancy CM: Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003, 290:1624-1632.
• [10]Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K: A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol 2009, 62:464-475.
• [11]Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K: A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. CMAJ 2009, 180:E47-E57.
• [12]Thorpe KE, Oxman AD, Treweek S, Furberg CD, Grp PW: Pragmatic trials are randomized and may use a placebo Reply. J Clin Epidemiol 2010, 63:694-695.
• [13]Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D: Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ 2008, 337:a2390.
• [14]Riddle DL, Johnson RE, Jensen MP, Keefe FJ, Kroenke K, Bair MJ, Ang DC: The Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) instrument was useful for refining a randomized trial design: Experiences from an investigative team. J Clin Epidemiol 2010, 63:1271-1275.
• [15]Tosh G, Soares-Weiser K, Adams CE: Pragmatic vs explanatory trials: the pragmascope tool to help measure differences in protocols of mental health randomized controlled trials. Dialogues Clin Neurosci 2011, 13:209-215.
• [16]Glasgow RE, Gaglio B, Bennett G, Jerome GJ, Yeh HC, Sarwer DB, Appel L, Colditz G, Wadden TA, Wells B: Applying the PRECIS criteria to describe three effectiveness trials of weight loss in obese patients with comorbid conditions. Health Serv Res 2012, 47:1051-1067.
• [17]Koppenaal T, Linmans J, Knottnerus JA, Spigt M: Pragmatic vs. explanatory: An adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice. J Clin Epidemiol 2011, 64:1095-1101.
• [18]Bratton DJ, Nunn AJ: Alternative approaches to tuberculosis treatment evaluation: the role of pragmatic trials. Int J Tuberc Lung Dis 2011, 15:440-446.
• [19]Witt CM, Manheimer E, Hammerschlag R, Ludtke R, Lao L, Tunis SR, Berman BM: How well do randomized trials inform decision making: systematic review using comparative effectiveness research measures on acupuncture for back pain. PLoS One 2012, 7:e32399.
• [20]Selby P, Brosky G, Oh PI, Raymond V, Ranger S: How pragmatic or explanatory is the randomized, controlled trial? The application and enhancement of the PRECIS tool to the evaluation of a smoking cessation trial. BMC Med Res Methodol 2012, 12:101.
• [21]Bratton DJ, Nunn AJ, Wojnarowska F, Kirtschig G, Sandell A, Williams HC: The value of the pragmatic-explanatory continuum indicator summary wheel in an ongoing study: the bullous pemphigoid steroids and tetracyclines study. Trials 2012, 13:50. BioMed Central Full Text
• [22]Hasson F, Keeney S, McKenna H: Research guidelines for the Delphi survey technique. J Adv Nurs 2000, 32:1008-1015.
• [23]Keeney S, Hasson F, McKenna H: Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs 2006, 53:205-212.
• [24]McKenna H: The Delphi technique: a worthwhile research approach for nursing? J Adv Nurs 1994, 19:1221-1225.
• [25]Hsu C-C, Sandford BA: The Delphi Technique: Making Sense of Consensus. Practical Assess Res Eval 2007, 12:1-8.
• [26]Treweek S, Barnett K, Maclennan G, Bonetti D, Eccles MP, Francis JJ, Jones C, Pitts NB, Ricketts IW, Weal M, Sullivan F: E-mail invitations to general practitioners were as effective as postal invitations and were more efficient. J Clin Epidemiol 2012, 65:793-797.
• [27]Rosenbaum SE, Glenton C, Nylund HK, Oxman AD: User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol 2010, 63:607-619.
• [28]Ware JH, Hamel MB: Pragmatic trials–guides to better patient care? N Engl J Med 2011, 364:1685-1687.
• [29]Godwin M, Ruhland L, Casson I, MacDonald S, Delva D, Birtwhistle R, Lam M, Seguin R: Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol 2003, 3:28. BioMed Central Full Text
• [30]Higgins J, Altman D, Gøtzsche P, Jüni P, Moher D, Oxman A, Savović J, Schulz K, Weeks L, Sterne J: The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011, 343:d5928.