The importance of rating scales in measuring patient-reported outcomes

Health and Quality of Life Outcomes

Konrad Pesudovs³  Ecosse L Lamoureux²  Colm McAlinden³  Vijaya K Gothwal¹  Jyoti Khadka³ 

[1] Meera and L B Deshpande Centre for Sight Enhancement, Vision Rehabilitation Centres, L V Prasad Eye Institute, Hyderabad, India;Singapore National Eye Centre, Singapore Eye Research Institute, Singapore, Singapore;NH & MRC Centre for Clinical Eye Research, Discipline of Optometry and Vision Science, Flinders Medical Centre and Flinders University of South Australia, Adelaide, South Australia, 5042, Australia

关键词: Rasch analysis;    Quality of life;    Response categories;    Rating scales;    Patient reported outcomes;   

Others  :  825418
DOI  :  10.1186/1477-7525-10-80

 received in 2012-02-11, accepted in 2012-06-27,  发布年份 2012

【 摘 要 】

Background

A critical component that influences the measurement properties of a patient-reported outcome (PRO) instrument is the rating scale. Yet, there is a lack of general consensus regarding optimal rating scale format, including aspects of question structure, the number and the labels of response categories. This study aims to explore the characteristics of rating scales that function well and those that do not, and thereby develop guidelines for formulating rating scales.

Methods

Seventeen existing PROs designed to measure vision-related quality of life dimensions were mailed for self-administration, in sets of 10, to patients who were on a waiting list for cataract extraction. These PROs included questions with ratings of difficulty, frequency, severity, and global ratings. Using Rasch analysis, performance of rating scales were assessed by examining hierarchical ordering (indicating categories are distinct from each other and follow a logical transition from lower to higher value), evenness (indicating relative utilization of categories), and range (indicating coverage of the attribute by the rating scale).

Results

The rating scales with complicated question format, a large number of response categories, or unlabelled categories, tended to be dysfunctional. Rating scales with five or fewer response categories tended to be functional. Most of the rating scales measuring difficulty performed well. The rating scales measuring frequency and severity demonstrated hierarchical ordering but the categories lacked even utilization.

Conclusion

Developers of PRO instruments should use a simple question format, fewer (four to five) and labelled response categories.

【 授权许可】

   
2012 Khadka et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

【 参考文献 】

• [1]Varma R, Richman EA, Ferris FL, Bressler NM: Use of patient-reported outcomes in medical product development: a report from the 2009 NEI/FDA clinical trial endpoints symposium. Investig Ophthalmol Vis Sci 2010, 51:6095-6103.
• [2]Fayers PM, Sprangers MA: Understanding self-rated health. Lancet 2002, 359:187-188.
• [3]Revicki DA, Cella DF: Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res 1997, 6:595-600.
• [4]Hobart JC, Cano SJ, Zajicek JP, Thompson AJ: Rating scales as outcome measures for clinical trials in neurology: problems, solutions, and recommendations. Lancet Neurol 2007, 6:1094-1105.
• [5]Revicki DA: FDA draft guidance and health-outcomes research. Lancet 2007, 369:540-542.
• [6]Massof RW: Likert and Guttman scaling of visual function rating scale questionnaires. Ophthalmic Epidemiol 2004, 11:381-399.
• [7]Linacre JM: Investigating rating scale category utility. J Outcome Meas 1999, 3:103-122.
• [8]Linacre JM: Optimizing rating scale category effectiveness. J Appl Meas 2002, 3:85-106.
• [9]Garratt AM, Helgeland J, Gulbrandsen P: Five-point scales outperform 10-point scales in a randomized comparison of item scaling for the Patient Experiences Questionnaire. J Clin Epidemiol 2011, 64:200-207.
• [10]Preston CC, Colman AM: Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychol (Amst) 2000, 104:1-15.
• [11]Viswanathan M, Begen M, Dutta S, Childers T: Does a single response category in a scale completely capture a response? Psychol Mark 1996, 13:457-479.
• [12]Cox E: The optimal number of response alternatives in a scale: a review. J Market Res Soc 1980, 22:29-43.
• [13]Rasch G: Probabilistic models for some intelligence and attainment tests. Copenhagen: Nielsen and Lydiche; 1960. 2nd edition, Chicago, University of Chicago Press, 1980
• [14]Mallinson T: Why measurement matters for measuring patient vision outcomes. Optom Vis Sci 2007, 84:675-682.
• [15]Hays RD: Item response theory models. In Quality of Life Assessment in Clinical Trials: Methods and Practice. Edited by Staquet MJ, Hays RD, Fayers PM. New York: Oxford University Press; 1998:183-190.
• [16]Tesio L: Measuring behaviours and perceptions: Rasch analysis as a tool for rehabilitation research. J Rehabil Med 2003, 35:105-115.
• [17]Andrich D, de Jong JHAL, Sheridan BE: Diagnostic opportunities with the Rasch model for ordered response categories. In Application of latent trait and latent class models in the social sciences. Edited by Rost J, Langeheine R. Munster: Waxmann; 1997.
• [18]Andrich D: A rating scale formulation for ordered response categories. Psychometrika 1978, 43:561-573.
• [19]Wright BD, Mok M: Rasch models overview. J Appl Meas 2000, 1:83-106.
• [20]Reeve BB, Fayers P: Applying item response theory modeling for evaluating questionnaire items and scale properties. 2nd edition. Oxford: Oxford University Press; 2005.
• [21]Khadka J, Ryan B, Margrain TH, Court H, Woodhouse JM: Development of the 25-item Cardiff Visual Ability Questionnaire for Children (CVAQC). Br J Ophthalmol 2010, 94:730-735.
• [22]Wright B, Linacre JM: Reasonable mean-square fit values. Rasch Meas Trans 1994, 8:370.
• [23]Gothwal VK, Wright T, Lamoureux EL, Pesudovs K: Psychometric properties of visual functioning index using Rasch analysis. Acta Ophthalmol 2010, 88:797-803.
• [24]Gothwal VK, Wright TA, Lamoureux EL, Khadka J, McAlinden C, Pesudovs K: Improvements in visual ability with first-eye, second-eye, and bilateral cataract surgery measured with the visual symptoms and quality of life questionnaire. J Cataract Refract Surg 2011, 37:1208-1216.
• [25]Gothwal VK, Wright TA, Lamoureux EL, Lundstrom M, Pesudovs K: Catquest questionnaire: re-validation in an Australian cataract population. Clin Exp Ophthalmol 2009, 37:785-794.
• [26]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Visual Activities Questionnaire: assessment of subscale validity for cataract surgery outcomes. J Cataract Refract Surg 2009, 35:1961-1969.
• [27]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Cataract symptom score questionnaire: Rasch revalidation. Ophthalmic Epidemiol 2009, 16:296-303.
• [28]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Rasch analysis of visual function and quality of life questionnaires. Optom Vis Sci 2009, 86:1160-1168.
• [29]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Cataract Symptom Scale: clarifying measurement. Br J Ophthalmol 2009, 93:1652-1656.
• [30]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Using Rasch analysis to revisit the validity of the Cataract TyPE Spec instrument for measuring cataract surgery outcomes. J Cataract Refract Surg 2009, 35:1509-1517.
• [31]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Measuring outcomes of cataract surgery using the Visual Function Index-14. J Cataract Refract Surg 2010, 36:1181-1188.
• [32]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Activities of daily vision scale: what do the subscales measure? Investig Ophthalmol Vis Sci 2010, 51:694-700.
• [33]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Multiplicative rating scales do not enable measurement of vision-related quality of life. Clin Exp Optom 2011, 94:52-62.
• [34]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: The impact of cataract surgery questionnaire: re-evaluation using Rasch analysis. Acta Ophthalmol 2011, 89:423-428.
• [35]Pesudovs K, Gothwal VK, Wright T, Lamoureux EL: Remediating serious flaws in the National Eye Institute Visual Function Questionnaire. J Cataract Refract Surg 2010, 36:718-732.
• [36]Pesudovs K, Wright TA, Gothwal VK: Visual disability assessment: valid measurement of activity limitation and mobility in cataract patients. Br J Ophthalmol 2010, 94:777-781.
• [37]Gothwal VK, Wright TA, Lamoureux EL, Pesudovs K: Rasch analysis of the quality of life and vision function questionnaire. Optom Vis Sci 2009, 86:E836-E844.
• [38]Pesudovs K, Caudle LE, Rees G, Lamoureux EL: Validity of a visual impairment questionnaire in measuring cataract surgery outcomes. J Cataract Refract Surg 2008, 34:925-933.
• [39]Linacre JM: A User's Guide to Winsteps: Rasch model Computer Programme. Chicago: Mesa Press; 2008.
• [40]Kirkwood BJ, Pesudovs K, Latimer P, Coster DJ: The efficacy of a nurse-led preoperative cataract assessment and postoperative care clinic. Med J Aust 2006, 184:278-281.
• [41]Mangione T: Mail surveys: improving the quality. Thousand Oaks: Sage; 1995.
• [42]Hjermstad MJ, Fayers PM, Haugen DF, Caraceni A, Hanks GW, Loge JH, Fainsinger R, Aass N, Kaasa S: Studies comparing Numerical Rating Scales, Verbal Rating Scales, and Visual Analogue Scales for assessment of pain intensity in adults: a systematic literature review. J Pain Symptom Manag 2011, 41:1073-1093.
• [43]Svensson E: Concordance between ratings using different scales for the same variable. Stat Med 2000, 19:3483-3496.
• [44]Hawthorne G, Mouthaan J, Forbes D, Novaco RW: Response categories and anger measurement: do fewer categories result in poorer measurement? development of the DAR5. Soc Psychiatry Psychiatr Epidemiol 2006, 41:164-172.
• [45]Pesudovs K, Coster DJ: An instrument for assessment of subjective visual disability in cataract patients. Br J Ophthalmol 1998, 82:617-624.
• [46]Hendriks AA, Vrielink MR, van Es SQ, De Haes HJ, Smets EM: Assessing inpatients’ satisfaction with hospital care: should we prefer evaluation or satisfaction ratings? Patient Educ Couns 2004, 55:142-146.
• [47]Hendriks AA, Vrielink MR, Smets EM, van Es SQ, De Haes JC: Improving the assessment of (in)patients’ satisfaction with hospital care. Med Care 2001, 39:270-283.
• [48]Lundstrom M, Pesudovs K: Catquest-9SF patient outcomes questionnaire: nine-item short-form Rasch-scaled revision of the Catquest questionnaire. J Cataract Refract Surg 2009, 35:504-513.