Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions

Trials

Julie Taylor⁴  Joanne Coyle²  Shaun Treweek⁵  Brian Williams³  Mary Wells¹ 

[1] School of Nursing and Midwifery, University of Dundee, 11 Airlie Place, Dundee, DD1 4HJ, UK;Social Dimensions of Health Institute, University of St Andrews and University of Dundee, 11 Airlie Place, Dundee, DD1 4HJ, UK;Nursing, Midwifery and Allied Health Professions Research Unit, Iris Murdoch Building, University of Stirling, Stirling, FK9 4LA, UK;National Society for the Prevention of Child Abuse and Neglect (NSPCC), Weston House, 42 Curtain Rd, London, EC2A 3NH, UK;Population Health Sciences, School of Medicine, University of Dundee, The Mackenzie building, Kirsty Semple Way, Dundee, DD2 4BF, UK

关键词: Complex interventions;    Context;    Case study;    RCT;   

Others  :  1095509
DOI  :  10.1186/1745-6215-13-95

 received in 2011-10-21, accepted in 2012-05-21,  发布年份 2012

【 摘 要 】

Background

A number of single case reports have suggested that the context within which intervention studies take place may challenge the assumptions that underpin randomised controlled trials (RCTs). However, the diverse ways in which context may challenge the central tenets of the RCT, and the degree to which this information is known to researchers or subsequently reported, has received much less attention. In this paper, we explore these issues by focusing on seven RCTs of interventions varying in type and degree of complexity, and across diverse contexts.

Methods

This in-depth multiple case study using interviews, focus groups and documentary analysis was conducted in two phases. In phase one, a RCT of a nurse-led intervention provided a single exploratory case and informed the design, sampling and data collection within the main study. Phase two consisted of a multiple explanatory case study covering a spectrum of trials of different types of complex intervention. A total of eighty-four data sources across the seven trials were accessed.

Results

We present consistent empirical evidence across all trials to indicate that four key elements of context (personal, organisational, trial and problem context) are crucial to understanding how a complex intervention works and to enable both assessments of internal validity and likely generalisability to other settings. The ways in which context challenged trial operation was often complex, idiosyncratic, and subtle; often falling outside of current trial reporting formats. However, information on such issues appeared to be available via first hand ‘insider accounts’ of each trial suggesting that improved reporting on the role of context is possible.

Conclusions

Sufficient detail about context needs to be understood and reported in RCTs of complex interventions, in order for the transferability of complex interventions to be assessed. Improved reporting formats that require and encourage the clarification of both general and project-specific threats to the likely internal and external validity need to be developed. In addition, a cultural change is required in which the open and honest reporting of such issues is seen as an indicator of study strength and researcher integrity, rather than a symbol of a poor quality study or investigator ability.

【 授权许可】

   
2012 Wells et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Medical Research Council: A framework for development and evaluation of RCTs for complex interventions to improve health. London: Medical Research Council; 2000.
• [2]Hoddinott P, Britten J, Pill R: Why do interventions work in some places and not others: a breastfeeding support group trial. Soc Sci Med 2010, 70:769-778.
• [3]Voigt-Radloff S, Graff M, Leonhart R, Hull M, Rikkert M, Vernooij-Dassen M: Why did an effective Dutch complex psycho-social intervention for people with dementia not work in the German healthcare context? Lessons learnt from a process evaluation alongside a multicentre RCT. BMJ Open 2011, 1:e000094.
• [4]Wilson DK, Griffin S, Saunders RP, Kitzman-Ulrich H, Meyers DC, Mansard L: Using process evaluation for program improvement in dose, fidelity and reach: the ACT trial experience. International Journal of Behavioural Nutrition and Physical Activity 2009, 6:79. BioMed Central Full Text
• [5]Plsek PE, Greenhalgh T: The challenge of complexity in health care. BMJ 2001, 323:625-628.
• [6]Blackwood B: Methodological issues in evaluating complex healthcare interventions. J Adv Nurs 2006, 54:612-622.
• [7]Craig P, Dieppe P, MacIntyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008, 337:a1655.
• [8]Hawe P, Shiell A, Riley T: Complex interventions: how "out of control" can a randomised controlled trial be? BMJ 2004, 328:1561-1563.
• [9]Perera R, Henghan C, Yudkin P: Graphical method for depicting randomised trials of complex interventions. BMJ 2007, 334:127-129.
• [10]Butler H, Bowes G, Drew S, Glover S, Godfrey C, Patton G, Trafford L, Bond L: Harnessing complexity: taking advantage of context and relationships in dissemination of school-based interventions. Health Promot Pract 2010, 11:259-267.
• [11]Iliffe S: Hospital at home: from red to amber? BMJ 1998, 316:1761-1762.
• [12]Dowswell G, Lawler J, Young J: Unpacking the 'black box' of a nurse-led stroke support service. Clin Rehabil 2000, 14:160-171.
• [13]Dowswell G, Lawler J, Dowswell T, Young J, Forster A, Hearn J: Investigating recovery from stroke: a qualitative study. J Clin Nurs 2000, 9:507-515.
• [14]van Meijel B, Gamel C, van Swieten-Duijfjes B, Grypdonck MH: The development of evidence-based nursing interventions: methodological considerations. J Adv Nurs 2004, 48:84-92.
• [15]Glaziou P, Meats E, Heneghan C, Shepperd S: What is missing from descriptions of treatment in trials and reviews? BMJ 2008, 336:1472-1474.
• [16]Karanicolas P, Montori V, Devereaux P, Schünemann H, Guyatt G: A new 'mechanistic-practical' framework for designing and interpreting randomized trials. J Clin Epidemiol 2009, 62:479-484.
• [17]Kent D, Kitsios G: Against pragmatism: on efficacy, effectiveness and the real world. Trials 2009, 10:48. BioMed Central Full Text
• [18]Oxman A, Lombard C, Treweek S, Gagnier J, Maclure M, Zwarenstein M: Why we will remain pragmatists: four problems with the impractical mechanistic framework and a better solution. J Clin Epidemiol 2009, 62:485-488.
• [19]ICEBeRG: Designing theoretically-informed implementation interventions. (The improved clinical effectiveness through behavioural research group). Implementation Science 2006, 1:4. BioMed Central Full Text
• [20]Connelly J: Realism in evidence based medicine: interpreting the randomised controlled trial. Journal of Health Organisation and Management 2004, 18:70-81.
• [21]McCormack J, Greenhalgh T: Seeing what you want to see in a randomised controlled trial: versions and perversions of UKPDS data. BMJ 2000, 320:1720-1723.
• [22]Pawson R, Greenhalgh T, Harvey G, Walshe K: Realist review - a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 2005, 10:21-34.
• [23]Boutron I, Moher D, Altman D, Schulz K, Ravaud P: Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med 2008, 148:295-309.
• [24]Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D, for the CONSORT and Pragmatic Trials in Healthcare (Practihc) groups: Improving the reporting of pragmatic trials: an extension of the CONSORT Statement. BMJ 2008., 337
• [25]Schulz R, Czaja SJ, McKay JR, Ory MG, Bell SH: Intervention Taxonomy (ITAX): describing essential features of interventions. American J of Health Behav 2010, 34:811-821.
• [26]Hart A: What is a research question? A case study in the early stages of design of a randomised controlled trial for a complementary therapy. Complement Ther Med 2003, 11:42-45.
• [27]Wight D, Obasi A: Unpacking the 'black box': the importance of process data to explain outcomes. In Effective Sexual Interventions: issues in experimental evaluation. Edited by Stephenson J, Imrie J, Bonell C. Oxford: Oxford University Press; 2003:151-169.
• [28]Toroyan T, Oakley A, Laing G, Roberts I, Mugford M, Turner J: The impact of day care on socially disadvantaged families: an example of the use of process evaluation within a randomized controlled trial. Care, Health & Development 2004, 30:691-698.
• [29]Wells EM: Behind the scenes of randomised trials of complex interventions: insiders reveal the importance of context. PhD Thesis. Dundee: College of Medicine, Dentistry and Nursing, University of Dundee; 2007.
• [30]Yin R: Case study research: Design and Methods. 2nd edition. Thousand Oaks: Sage; 1994.
• [31]Stoecker R: Evaluating and rethinking the case study. Sociol Rev 1991, 39:88-112.
• [32]Crowe C, Cresswell K, Robertson R, Huby G, Avery A, Sheikh A: The case study approach. BMC Research Methodology 2011, 11:100. BioMed Central Full Text
• [33]Wells M, Harrow A, Donnan P, Davey P, Devereux S, Little G, McKenna E, Wood R, Chen R, Thompson A: Patient, carer and health service outcomes of nurse-led early discharge after breast cancer surgery: a randomised controlled trial. Br J Cancer 2004, 91:651-658.
• [34]National Research Register (NRR) Archive http://www.nihr.ac.uk/Pages/NRRArchive.aspx webcite
• [35]Ritchie J, Spencer L: Qualitative data analysis for applied policy research. In Analysing Qualitative Data. Edited by Bryman A, Burgess R. London: Routledge; 1993:173-194.
• [36]NVIVO 2 http://www.qsrinternational.com/products_previous-products_nvivo2.aspx webcite
• [37]Mays N: Qualitative Research: rigour and qualitative research. BMJ 1995, 311:109-112.
• [38]Mays N, Pope C: Assessing quality in qualitative research. BMJ 2000, 320:50-52.
• [39]Kovach C, Cashin J, Sauer L: Deconstruction of a complex tailored intervention to assess and treat discomfort of people with advanced dementia. J Adv Nurs 2006, 55:678-688.
• [40]Abraham C, Michie S: A taxonomy of behavior change techniques used in interventions. Health psychology: Official journal of the Division of Health Psychology, American Psychological Association 2008, 27:379-387.
• [41]Voigt-Radloff S, Graff M, Leonhart R, Hull M, Rikkert MO, Vernooij-Dassen M: Why did an effective Dutch complex psycho-social intervention for people with dementia not work in the German healthcare context? Lessons learnt from a process evaluation alongside a multicentre RCT. BMJ Open 2011.
• [42]Mcdonald A, Treweek S, Shakur H, Free C, Knight R, Speed C, Campbell M: Using a business model approach and marketing techniques for recruitment to clinical trials. Trials 2011, 12:74. BioMed Central Full Text
• [43]Czaja SJ, Schulz R, Lee CC, Belle SH, REACH Investigators: A methodology for describing and decomposing complex psychosocial and behavioural interventions. Psychol Aging 2003, 18:385-395.
• [44]Connell J, Kubisch A: Applying a theory of change approach to the evaluation of comprehensive community initiatives: progress, prospects, and problems. http://www.aspeninstitute.org/site/c.huLWJeMRKpH/b.613709/k.B547/Applying_a_Theory_of_Change_Approach_to_the_Evaluation_of_Comprehensive_Community_Initiatives_Progress_Prospects_and_Problems.htm webcite
• [45]Glasgow RE: Evaluation of theory-based interventions: The RE-AIM model. In Health Behavior and Health Education. 3rd edition. Edited by Glanz K, Lewis FM, Rimer BK. San Francisco: Jossey-Bass; 2002:531-544.
• [46]Lloyd JJ, Logan S, Greaves CJ, Wyatt KM: Evidence, theory and context - using intervention mapping to develop a school-based intervention to prevent obesity in children. Int J Behav Nutr Phys Act 2011, 8:73. BioMed Central Full Text
• [47]Hardeman W, Sutton S, Griffin S, Johnston M, White A, Wareham NJ, Kinmonth AL: A causal modelling approach to the development of theory-based behaviour change programmes for trial evaluation. Health Educ Res 2005, 20:676-687.
• [48]Fayter D, McDaid C, Ritchie G, Stirk L, Eastwood A: Systematic review of barriers, modifiers and benefits involved in participation in cancer clinical trials. Centre for Reviews and Dissemination. York: University of York; 2006. http://www.york.ac.uk/inst/crd/CRD_Reports/crdreport31.pdf webcite
• [49]Ziebland S, Featherstone K, Snowdon C, Barker K, Frost H, Fairbank J: Does it matter if clinicians recruiting for a trial don't understand what the trial is really about? Qualitative study of surgeons experiences of participation in a pragmatic multi-centre RCT. Trials 2007, 8:4. BioMed Central Full Text
• [50]Snowdon C, Garcia J, Elbourne D: Making sense of randomisation; responses of parents of critically ill babies to random allocation of treatment in a clinical trial. Soc Sci Med 1997, 45:1337-1355.
• [51]Dolan G, Broomfield J, Lewith G, Watkins A: Operational and resource management of an RCT: some of the pitfalls and lessons learned. J Clin Nurs 1999, 8:389-395.
• [52]Treweek S, Mitchell E, Pitkethly M, Cook J, Kjeldstrøm M, Johansen M, Taskila T, Sullivan F, Wilson S, Jackson C, Jones R, Lockhart P: Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev 2010, 4:MR000013.
• [53]McKinstry B, Hammersley V, Daly F, Sullivan F: Recruitment and retention in a multicentre randomised controlled trial in Bells palsy: A case study. BMC Med Res Methodol 2007, 7:15. BioMed Central Full Text
• [54]Prescott R, Counsell C, Gillespie W, Grant A, Russell I, Kiauka S, Colthart I, Ross S, Shepherd S, Russell D: Factors that limit the quality, number and progress of randomised controlled trials. Health Technol Assess 1999, 3:1-143.
• [55]Campbell M, Snowdon C, Francis D, Elbourne D, McDonald A, Knight R, Entwistle V, Garcia J, Roberts I, Grant A, Grant A, STEPS group: Recruitment to randomised trials: strategies for trial enrolment and participation study. The STEPS study. Health Technol Assess 2007., 11http://www.hta.ac.uk/fullmono/mon1148.pdf webcite
• [56]Plant H, Bredin M, Krishnasamy M, Corner J: Working with resistance, tension and objectivity: conducting a randomised controlled trial of a nursing intervention for breathlessness. NT Research 2000, 5:426-434.
• [57]Bredin M, Corner J, Krishnasamy M, A'Hern R, Bailey C: Multicentre randomised controlled trial of nursing intervention for breathlessness in patients with lung cancer. BMJ 1999, 318:901-904.
• [58]Schulz KF: Unbiased research and the human spirit: the challenges of randomized controlled trials. Can Med Assoc J 1995, 153:783-786.
• [59]Watson J, Torgerson D: Increasing recruitment to randomised trials: a review of randomised controlled trials. BMC Med Res Methodol 2006, 6:34. BioMed Central Full Text
• [60]Lawton J, Jenkins N, Darbyshire J, Holman R, Farmer A, Hallowell N: Challenges of maintaining research protocol fidelity in a clinical care setting: A qualitative study of the experiences and views of patients and staff participating in a randomized controlled trial. Trials 2011, 12:108. BioMed Central Full Text
• [61]Schulz KF, Altman DG, Moher D, CONSORT Group: CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. Trials 2010, 11:32. BioMed Central Full Text
• [62]Mackenzie M, O'Donnell C, Halliday E, Sridharan S, Platt S: Evaluating complex interventions: one size does not fit all. BMJ 2010, 340:401-403.
• [63]Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008, 8:45. BioMed Central Full Text
• [64]Power R, Williams B: Checklists for improving rigour in qualitative research. Including personal reflections might help. BMJ 2001, 323:514.3.
• [65]Kingdon C: Reflexivity: Not just a qualitative methodological research tool. British Journal of Midwifery 2005, 13:622-627.
• [66]MacPherson H, Thorpe L, Thomas K: Beyond needling - therapeutic processes in acupuncture care: a qualitative study nested within a low-back pain trial. J Altern Complement Med 2006, 12:873-880.
• [67]Finlay L, Gough B (Eds): Reflexivity: a practical guide for researchers in health and social science. Oxford: Blackwell; 2003.
• [68]Audrey S, Holliday J, Parry-Langdon N, Campbell R: Meeting the challenges of implementing process evaluation within randomized controlled trials: the example of ASSIST (A Stop Smoking in Schools Trial). Health Educ Res 2006, 21:366-377.
• [69]Hotopf M, Churchill R, Lewis G: Pragmatic randomised controlled trials in psychiatry. Br J Psychiatry 1999, 175:217-223.
• [70]Oakley A, Strange V, Bonell C, Allen E, Stephenson J, Ripple Study Team: Process evaluation in randomised controlled trials of complex interventions. BMJ 2006, 332:413-416.
• [71]Schwartz D, Lelouch J: Explanatory and pragmatic attitudes in therapeutic trials. Journal of Chronic Disease 1967, 20:637-648.
• [72]Godwin M, Ruhland L, Casson I, MacDonald S, Delva D, Birtwhistle R, Lam M, Sequin R: Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol 2003, 3:28. BioMed Central Full Text
• [73]Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, Finch T, Kennedy A, Mair F, O’Donnell K, Ong BN, Rapley T, Rogers A, May C: Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med 2010, 8:63. BioMed Central Full Text
• [74]Riley T, Hawe P, Shiell A: Contested ground: how should qualitative evidence inform the conduct of a community intervention trial? J Health Serv Res Policy 2005, 10:103-110.
• [75]McCormack B, Kitson A, Harvery G, Rycroft-Malone J, Titchen A, Seers K: Getting evidence into practice: the meaning of 'context'. J Adv Nurs 2002, 38:94-104.