An administrative data validation study of the accuracy of algorithms for identifying rheumatoid arthritis: the influence of the reference standard on algorithm performance

BMC Musculoskeletal Disorders

Karen Tu⁵  J Carter Thorne¹  R Liisa Jaakkimainen²  Debra A Butt⁷  Noah Ivers⁶  Jacqueline Young⁵  Diane Green⁵  J Michael Paterson⁴  Sasha Bernatsky³  Claire Bombardier⁷  Jessica Widdifield⁷ 

[1] Southlake Regional Health Centre, Newmarket, ON, Canada;Sunnybrook Health Sciences Centre, Toronto, ON, Canada;McGill University, Montreal, QC, Canada;McMaster University, Hamilton, ON, Canada;Institute for Clinical Evaluative Sciences, Toronto, ON, Canada;Women’s College Hospital, Toronto, ON, Canada;University of Toronto, Toronto, 200 Elizabeth St 13EN-224, Toronto, ON M5G 2C4, Canada

关键词: Diagnostic test;    Predictive values;    Sensitivity and specificity;    Validation study;    Health administrative databases;    Rheumatoid arthritis;   

Others  :  1125518
DOI  :  10.1186/1471-2474-15-216

 received in 2013-12-16, accepted in 2014-06-10,  发布年份 2014

【 摘 要 】

Background

We have previously validated administrative data algorithms to identify patients with rheumatoid arthritis (RA) using rheumatology clinic records as the reference standard. Here we reassessed the accuracy of the algorithms using primary care records as the reference standard.

Methods

We performed a retrospective chart abstraction study using a random sample of 7500 adult patients under the care of 83 family physicians contributing to the Electronic Medical Record Administrative data Linked Database (EMRALD) in Ontario, Canada. Using physician-reported diagnoses as the reference standard, we computed and compared the sensitivity, specificity, and predictive values for over 100 administrative data algorithms for RA case ascertainment.

Results

We identified 69 patients with RA for a lifetime RA prevalence of 0.9%. All algorithms had excellent specificity (>97%). However, sensitivity varied (75-90%) among physician billing algorithms. Despite the low prevalence of RA, most algorithms had adequate positive predictive value (PPV; 51-83%). The algorithm of “[1 hospitalization RA diagnosis code] or [3 physician RA diagnosis codes with ≥1 by a specialist over 2 years]” had a sensitivity of 78% (95% CI 69–88), specificity of 100% (95% CI 100–100), PPV of 78% (95% CI 69–88) and NPV of 100% (95% CI 100–100).

Conclusions

Administrative data algorithms for detecting RA patients achieved a high degree of accuracy amongst the general population. However, results varied slightly from our previous report, which can be attributed to differences in the reference standards with respect to disease prevalence, spectrum of disease, and type of comparator group.

【 授权许可】

   
2014 Widdifield et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

【 参考文献 】

• [1]Bernatsky S, Lix L, O’Donnell S, Lacaille D, CANRAD Network: Consensus statements for the use of administrative health data in rheumatic disease research and surveillance. J Rheumatol 2013, 40:66-73.
• [2]Widdifield J, Labrecque J, Lix L, Paterson JM, Bernatsky S, Tu K, Ivers N, Bombardier C: A systematic review and critical appraisal of validation studies to identify rheumatic diseases in health administrative databases. Arthritis Care Res 2013, 65:1490-1503.
• [3]Benchimol EI, Manuel DG, To T, Griffiths AM, Rabeneck L, Guttmann A: Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol 2011, 64:821-829.
• [4]Widdifield J, Bernatsky S, Paterson JM, Tu K, Ng R, Thorne JC, Pope JE, Bombardier C: Accuracy of Canadian health administrative databases in identifying patients with rheumatoid arthritis: a validation study using the medical records of rheumatologists. Arthritis Care Res (Hoboken) 2013, 65:1582-1591.
• [5]Tu K, Mitiku TF, Ivers NM, Guo H, Lu H, Jaakkimainen L, Kavanagh DG, Lee DS, Tu JV: Evaluation of Electronic Medical Record Administrative data Linked Database (EMRALD). Am J Manag Care 2014, 20:e15-e21.
• [6]Arnett FC, Edworthy SM, Bloch DA, Mcshane DJ, Fries JF, Cooper NS, Healey LA, Kaplan SR, Liang MH, Luthra HS, Medsger TA, Mitchell DM, Neustadt DH, Pinals RS, Schaller JG, Sharp JT, Wilder RL, Hunder GG: The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum 1988, 31:315-324.
• [7]Aletaha D, Neogi T, Silman AJ, Funovits J, Felson DT, Bingham CO 3rd, Birnbaum NS, Burmester GR, Bykerk VP, Cohen MD, Combe B, Costenbader KH, Dougados M, Emery P, Ferraccioli G, Hazes JM, Hobbs K, Huizinga TW, Kavanaugh A, Kay J, Kvien TK, Laing T, Mease P, Ménard HA, Moreland LW, Naden RL, Pincus T, Smolen JS, Stanislawska-Biernat E, Symmons D, et al.: 2010 Rheumatoid arthritis classification criteria: an American College of Rheumatology/European League Against Rheumatism collaborative initiative. Arthritis Rheum 2010, 62:2569-2581.
• [8]Ontario Health Insurance Plan: The Program. Toronto: Ontario Ministry of Health and Long-Term Care; 2008. http://www.health.gov.on.ca/en/public/programs/ohip/ webcite
• [9]World Health Organization: International Classification of Diseases, Ninth Revision Clinical Modification. Geneva: World Health Organization; 1988.
• [10]CIHI: Canadian Institute for Health Information, The CIHI Data Quality Framework. Ottawa, Ont: CIHI; 2009.
• [11]Ontario Drug Benefit: The Program. Toronto: Ontario Ministry of Health and Long-Term Care; 2008. http://www.health.gov.on.ca/en/pro/programs/drugs/odbf_mn.aspx webcite. (Accessed April 2012)
• [12]Chan B, Schultz S: Supply and Utilization of General Practitioner and Family Physician Services in Ontario. Institute for Clinical Evaluative Sciences: Toronto, ON; 2005.
• [13]Improving Health Care Data in Ontario. ICES Investigative Report. Toronto: Institute for Clinical Evaluative Sciences; 2005.
• [14]Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Moher D, Rennie D, de Vet HC, Lijmer JG, Standards for Reporting of Diagnostic Accuracy Group: Standards for Reporting of Diagnostic Accuracy. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 2003, 138:W1-W12.
• [15]Crowson CS, Matteson EL, Myasoedova E, Michet CJ, Ernste FC, Warrington KJ, Davis JM 3rd, Hunder GG, Therneau TM, Gabriel SE: The lifetime risk of adult-onset rheumatoid arthritis and other inflammatory autoimmune rheumatic diseases. Arthritis Rheum 2011, 63:633-639.
• [16]Koepsell T, Weiss N: Epidemiologic Methods: Studying the Occurrence of Illness. New York, NY: Oxford University Press; 2003.
• [17]Hux JE, Tang M: Patterns of Prevalence and Incidence of Diabetes. Toronto: Institute for Clinical Evaluative Sciences; 2003.
• [18]Tu K, Campbell NR, Chen ZL, Cauch-Dudek KJ, McAlister FA: Accuracy of administrative databases in identifying patients with hypertension. Open Med 2007, 1:e18-e26.
• [19]Thomas SL, Edwards CJ, Smeeth L, Cooper C, Hall AJ: How accurate are diagnoses for rheumatoid arthritis and juvenile idiopathic arthritis in the general practice research database? Arthritis Rheum 2008, 59:1314-1321.
• [20]Glazier R, Zagorski B, Rayner J: Comparison of Primary Care Models in Ontario by Demographics, Case Mix and Emergency Department Use, 2008/09 to 2009/10. Toronto: Institute for Clinical Evaluative Sciences; 2012.
• [21]Kralj B, Kantarevic J: Primary Care in Ontario: Reforms, Investments and Achievements. Ontario Medical Review; 2012. Available online: http://www.oma.org/Resources/Documents/PrimaryCareFeature.pdf webcite
• [22]MD Physician Services Inc: Canadian Medical Association: Physician Remuneration Options. Ottawa, ON, Canada; 2011. Available Online: md.cma.ca
• [23]Chubak J, Pocobelli G, Weiss NS: Tradeoffs between accuracy measures for electronic health care data algorithms. J Clin Epidemiol 2012, 65:343-349. e2