A randomized clinical trial using an educational intervention demonstrated no effect on interobserver agreement on assessments of functional status

Archives of Public Health

L Paquay³  J De Lepeleire³  K Milisen⁴  M Ylieff¹  F Buntinx² 

[1] Université de Liège, Faculty of Psychology and Educational Sciences, Liège, Belgium

[2] Universiteit Maastricht, Department of General Practice, Maastricht, The Netherlands

[3] Katholieke Universiteit Leuven, Department of General Practice, Leuven, Belgium

[4] University Hospitals of Leuven, Department of Geriatric Medicine, Leuven, Belgium

关键词: Autonomie Gérontologie Groupe Iso-Ressources (AGGIR);    Belgian Evaluation Scale;    randomized clinical trial;    reliability;    nursing home;    nursing;    Assessment;   

Others  :  1083921
DOI  :  10.1186/0778-7367-67-3-128

【 摘 要 】

Aim

To evaluate the effect of an educational intervention on interobserver agreement of assess-ments of functional status performed by registered nurses and care assistants in a nursing home and to compare interobserver agreement in persons with and without cognitive impairment.

Background

High accuracy of assessments of functional status in care settings for older persons is needed for the efficacy of the planning and the evaluation of the nursing care.

Method

Randomized clinical trial. Six registered nurses and six care assistants were randomized to participate in an educational session about assessment instruments for functional status (intervention) or in a session about falls in the elderly (control). Each of the registered nurses and care assistants performed assessments on the same thirty-four residents using the Belgian Evaluation Scale (BES) and the AGGIR instrument. The kappa statistic (κ) for multiple observers (and its 95% confidence interval) was the main outcome measure.

Findings

At baseline, interobserver agreement for BES total score was: κ = 0.43 (0.35-0.51) in the intervention group and κ = 0.48 (0.39-0.57) in the control group. At the second assessment, agreement measures were: κ = 0.48 (0.41-0.57) in the intervention group and κ = 0.58 (0.50-0.66) in the control group. Results for AGGIR total scores were similar.

Conclusion

Interobserver agreement of assessments on nursing home residents was moderate and did not improve significantly after an educational session.

【 授权许可】

   
2009 Paquay et al.

【 图 表 】

Figure 1.

Figure 2.

【 参考文献 】

• [1]Morrissey Y, Bowman C, Carpenter I: Assessment of patients in long-term care should be used to improve quality as well as allocate funds. Editorial Age and Ageing 2006, 35:212-4.
• [2]Polit DF, Hungler BP: Essentials of nursing research. Philadelphia, J.B. Lippincott Company; 1995.
• [3]Cohen J: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 1960, XX:37-46.
• [4]Sim J, Wright CC: The Kappa Statistic in Reliability Studies: Use, Interpretation, and Sample Size Requirements. Physical Therapy 2005, 85:257-68.
• [5]Streiner DL, Norman GR: Health measurement scales. In A practical guide to their development and use. Third edition. Oxford, Oxford University Press; 2003.
• [6]Phillips CD, Chu CW, Morris JN, Hawes C: Effects of Cognitive Impairment on the Reliability of Geriatric Assessments in Nursing Homes. Journal of the American Geriatrics Society 1993, 41:136-42.
• [7]Cronin-Stubbs D, Swanson B, Dean-Baar S, Sheldon JA, Duchene P: The Effects of a Training Program on Nurses' Functional Performance Assessments. Applied Nursing Research 1992, 5(1):38-43.
• [8]Polderman KH, Jorna EMF, Girbes ARJ: Inter-observer variability in APACHE II scoring: effect of strict guidelines and training. Intensive Care Medicine 2001, 27:1365-9.
• [9]Yekrangi-Hartmann C, Bernhardt T, Baltissen R: Ratertraining to Establish Interrater Reliability in a Study with Alzheimer Patients [Trainingsmassnahmen zur Verbesserung der Interrater-Reliabilität in einer Alzheimer-Studie]. Zeitschrift für Gerontopsychologie & Geriatrie 1999, 12(3):143-55.
• [10]Arnaert A, Delesie L: Calibration of measurement data: the Belgian Institute of Health Insurance Index of ADL. [IJking van meetgegevens: RIZIV A.D.L.-index]. Acta Hospitalia 1999, 39(4):19-31.
• [11]Katz S, Ford A, Moskowitz R, Jackson B, Jaffe M: Studies of Illness in the Aged. JAMA 1963, 185:914-9.
• [12]Rijksinstituut voor Ziekte- en Invaliditeitsverzekering/Institut National d'Assurance Maladie-Invalidité (RIZIV/INAMI) Royal Decree of August 21, 2008 and information report on control procedures of the Belgian Evaluation Scale. [Koninklijk besluit van 21 augustus 2008 over de nieuwe "Kappa"-controles in de ROB's en RVT's. Nieuwe Brochure over de "Kappa"-controles in de ROB's en RVT's (vanaf 1 september 2008)] Website documents, available on: http://www.riziv.fgov.be/care/nl/residential-care/specific-information/katz.htm webcite (Website consulted February 28, 2009)
• [13]Vetel J-M, Leroux R, Ducoudray J-M, Prévost P: AGGIR, Precisions on it's origin, practical instructions for use. [AGGIR, Précisions sur sa genèse, Conseils pratiques d'utilisation]. La Revue du Généraliste et de la Gérontologie 1998, 47:27-33.
• [14]Lawton MP, Brody EM: Assessment of older people: Self-maintaining and instrumental activities of daily living. Gerontologist 1969, 9(3):179-86.
• [15]Folstein MF, Folstein SE, McHugh PR: 'Mini-Mental State': a practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research 1975, 12:189-98.
• [16]Roth M, Tym E, Mountjoy CQ, Huppert FA, Hendrie H, Verma S, Goddard R: CAMDEX - A standard instrument for the diagnosis of mental disorder in the elderly with special reference to the early detection of dementia. British Journal of Psychiatry 1986, 149:698-709.
• [17]Paquay L, De Lepeleire J, Schoenmakers B, Stessens J, Bouwen A, van der Burg M, Di Notte D, Gazon R, Ylieff M, Fontaine O, Buntinx F: The Qualidem project in Belgium. A two-center study on care needs and provision in dementia care: Inclusion criteria and description of the population. Archives of Public Health 2004, 62:145-62.
• [18]De Lepeleire J, Paquay L, Jacobs M: De verschillende schalen voor ADL-activiteiten voor volwassenen in de Vlaamse gezondheidszorg. Huisarts Nu 2005, 34(2):58-68.
• [19]Leroux R, Vetel J-M, Ducoudray J-M: AGGIR. Videotape. France, Syndicat National de Gérontologie Clinique; 1999.
• [20]Tombaugh TN, McIntyre NJ: The Mini Mental State Examination: A comprehensive review. Journal of the American Geriatrics Society 1992, 40:922-35.
• [21]Siegel S, Castellan NJ: Nonparametric statistics for the Behavioral Sciences. New York, Mc Graw-Hill; 1988.
• [22]Fleiss JL, Levin B, Paik MC: Statistical Methods for Rates and Proportions. Third edition. Hoboken NJ, John Wiley & Sons Inc.; 2003.
• [23]Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics 1977, 33:159-74.
• [24]Feinstein AR, Cicchetti DV: High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology 1990, 43:543-9.
• [25]Hoehler FK: Bias and prevalence effects on kappa viewed in terms of sensitivity and specificity. Journal of Clinical Epidemiology 2000, 53:499-503.
• [26]Tooth LR, Ottenbacher KJ: The κ Statistic in Rehabilitation Research: An Examination. Archives of Physical and Medical Rehabilitation 2004, 85:1371-6.
• [27]Dawson P, Wells D, Reid D, Sidani S: An abilities assessment instrument for elderly persons with cognitive impairment: psychometric properties and clinical utility. Journal of Nursing Measurement 1998, 6:35-54.
• [28]Volicer L, Hurley A, Lathi D, Kowall N: Measurement of severity in advanced Alzheimer's disease. Journal of Gerontology 1994, 49:M223-6.
• [29]Bellelli G, Frisoni G, Bianchetti A, Trabucchi M: The Bedford Alzheimer Nursing Severity Scale for the Severely Demented: Validation Study. Alzheimer Disease and Associated Disorders 1997, 11(2):71-7.
• [30]Lang MM: Screening for cognitive impairment in the older adult. The Nurse Practitioner 2001, 26(11):26. 32-4, 36-7,41