A case study of binary outcome data extraction across three systematic reviews of hip arthroplasty: errors and differences of selection

BMC Research Notes

Eva Kaltenthaler¹  Alison Scope¹  Christopher Carroll¹ 

[1] Health Economics and Decision Science (HEDS), School of Health and Related Research (ScHARR), Regent court, Regent Street, Sheffield S1 4DA, UK

关键词: Reporting;    Data-handling error;    Data extraction;    Meta-analysis;    Data interpretation;    Systematic review;   

Others  :  1138249
DOI  :  10.1186/1756-0500-6-539

 received in 2013-05-02, accepted in 2013-11-15,  发布年份 2013

【 摘 要 】

Background

Data extraction is a key stage in systematic review, yet it is the subject of little research. The aim of the present research was to use a small case study to highlight some important issues affecting this fundamental process.

Methods

The authors undertook an analysis of differences in the binary event data extracted and analysed by three systematic reviews on the same topic: a comparison of total hip arthroplasty and hemiarthroplasty. The following binary event data were extracted for three key outcomes, common to all three reviews, from those trials common to all three reviews: Dislocation rates, 1-year mortality, and revision rates. Differences between the data extracted by the three reviews were categorised as either errors or an issue of data selection. Meta-analysis was performed to assess whether these differences led to differences in summary estimates of effect.

Results

Across the three outcomes, differences in selection accounted for between 8% and 42% of the data differences between reviews, and errors accounted for between 8% and 17%. No rationale was given in any of these former cases for the choice of event data being reported. These differences did lead to small differences in meta-analysed relative risks between the two treatments in the three reviews, but none was significant.

Conclusions

Systematic reviewers should use double-data extraction to minimise error and also make every effort to clarify or explain their choice of data, within the scope of their publication. Reviewers frequently exercise selection when faced with a choice of alternative but potentially equally appropriate data for an outcome. However, this selection is rarely made clear by review authors. Systematic review was developed as a method specifically to be both reproducible and transparent. This case study suggests that neither objective is always being achieved.

【 授权许可】

   
2013 Carroll et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Horton J, Vandermeer B, Hartling L, Tjosvold L, Klassen T, Buscemi N: Systematic review data extraction: cross-sectional study showed that experience did not increase accuracy. J Clin Epidemiol 2010, 63:289-298.
• [2]Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen T: Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol 2006, 59:697-703.
• [3]Goetzsche P, Hr¢bjartsson A, Maric K, Tendal B: Data extraction errors in meta-analyses that use Standardized Mean Differences. JAMA 2007, 298:430-437.
• [4]Carroll C, Stevenson M, Scope A, Evans P, Buckley S: Hemiarthroplasty and total-hip arthroplasty for treating primary intracapsular fracture of the hip: A systematic review and cost effectiveness analyses. Health Technol Assess 2011, 15(36):1-7.
• [5]Parker MJ, Gurusamy KS, Azegami S: Arthroplasties (with and without bone cement) for proximal femoral fractures in adults. [Update of Cochrane Database Syst Rev. 2006;3:CD001706; PMID: 16855974]. Cochrane Database Syst Rev 2010, 6:CD001706.
• [6]Hopley C, Stengel D, Ekkernkamp A, Wich M: Primary total hip arthroplasty versus hemiarthroplasty for displaced intracapsular hip fractures in older patients: systematic review. Br Med J 2010, 340:c2332.
• [7]Liang LZ: Meta analysis of total hip arthroplasty versus hemiarthroplasty for displaced femoral neck fractures in elderly patients. J Clin Rehab Tissue Eng Res 2010, 14:3991-3995.
• [8]Mouzopoulos G, Stamatakos M, Arabatzi H, Vasiliadis G, Batanis G, Tsembeli A, et al.: The four-year functional result after a displaced subcapital hip fracture treated with three different surgical options. Int Orthop 2008, 32:367-373.
• [9]Blomfeldt R, Törnkvist H, Ponzer S, Söderqvist A, Tidermark J: Internal fixation versus hemiarthroplasty for displaced fractures of the femoral neck in elderly patients with severe cognitive impairment. J Bone Joint Surg Br Vol 2005, 87:523-529.
• [10]Carroll C, Scope A, Kaltenthaler E, Stevenson M: Re: Extraction and analysis errors and queries: Author Response. Br Med J 2010, 340:c2332.
• [11]Keating JF, Grant A, Masson M, Scott NW, Forbes JF: Randomized comparison of reduction and fixation, bipolar hemiarthroplasty, and total hip arthroplasty. Treatment of displaced intracapsular hip fractures in healthy older patients. J Bone Joint Surg Am Vol 2006, 88:249-260.
• [12]Baker RP, Squires B, Gargan MF, Bannister GC: Total hip arthroplasty and hemiarthroplasty in mobile, independent patients with a displaced intracapsular fracture of the femoral neck. A randomized, controlled trial. J Bone Joint Surg Am Vol 2006, 88:2583-2589.
• [13]Fergusson D, Aaron S, Guyatt GH, Hebert P: Post-randomisation exclusions: the intention to treat principle and excluding patients from analysis. Br Med J 2002, 325:652-654.
• [14]Higgins J: Measuring inconsistency in meta-analyses. Br Med J 2003, 327:557-560.
• [15]Guyatt GH, Oxman A, Vist G, Kunz R, Brozel J, Alonso-Coello P: GRADE guidelines : 4. Rating the quality of evidence - study limitations (risk of bias). J Clin Epidemiol 2011, 64:407-415.
• [16]Egger M, Smith G, Altman D: Systematic reviews in health care: meta-analysis in context. 2nd edition. London: BMJ Publishing Group; 2001.
• [17]Skinner P, Riley D, Ellery J, Beaumont A, Coumine R, Shafighian B: Displaced subcapital fractures of the femur: a prospective randomized comparison of internal fixation, hemiarthroplasty and total hip replacement. Injury 1989, 20:291-293.
• [18]Ravikumar KJ, Marsh G: Internal fixation versus hemiarthroplasty versus total hip arthroplasty for displaced subcapital fractures of femur–13 year results of a prospective randomised study. Injury 2000, 31:793-797.
• [19]Jones A, Remmington T, Williamson P, Ashley D, Smyth R: High prevalence but low impact of data extraction and reporting errors were found in Cochrane systematic reviews. J Clin Epidemiol 2005, 58:741-742.
• [20]Higgins J, Green S: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011. Available from http://www.cochrane-handbook.org webcite
• [21]Rotstein D, Laupacis A: Differences between systematic reviews and health technology assessments: a trade-off between the ideals of scientific rigor and the realities of policy making. Int J Technol Assess Health Care 2004, 20:177-183.
• [22]Dorr LD, Glousman R, Hoy AL, Vanis R, Chandler R: Treatment of femoral neck fractures with total hip replacement versus cemented and noncemented hemiarthroplasty. J Arthroplasty 1986, 1:21-28.
• [23]Macaulay W, Nellans KW, Garvin KL, Iorio R, Healy WL, Rosenwasser MP, et al.: Prospective randomized clinical trial comparing hemiarthroplasty to total hip arthroplasty in the treatment of displaced femoral neck fractures: winner of the Dorr Award. J Arthroplasty 2008, 23:2-8.