InterObserver Reliability Assessments in Time Motion Studies: TheFoundation for Meaningful Clinical Workflow Analysis

AMIA 2013 Annual Symposium

Marcelo A. Lopetegui1 ; MD ; Shasha Bai2 ; MS ; Po-Yin Yen1 ; RN ; PhD ; Albert Lai1 ; PhD

PID  :  122608

来源: CEUR

【 摘 要 】

Understanding clinical workflow is critical for researchers and healthcare decision makers. Currentworkflow studies tend to oversimplify and underrepresent the complexity of clinical workflow. Continuousobservation time motion studies (TMS) could enhance clinical workflow studies by providing rich quantitative datarequired for indepth workflow analyses. However, methodological inconsistencies have been reported in continuousobservation TMS, potentially reducing the validity of TMS’ data and limiting their contribution to the general stateof knowledge. We believe that a cornerstone in standardizing TMS is to ensure the reliability of the humanobservers. In this manuscript we review the approaches for interobserver reliability assessment (IORA) in arepresentative sample of TMS focusing on clinical workflow. We found that IORA is an uncommon practice,inconsistently reported, and often uses methods that provide partial and overestimated measures of agreement. Sincea comprehensive approach to IORA is yet to be proposed and validated, we provide initial recommendations forIORA reporting in continuous observation TMS.BACKGROUND AND RATIONALEUnderstanding the complexity of clinical workflow provides researchers and administrators with thenecessary knowledge to assess some of the most critical issues in healthcare, such as: increasing patient safety bydetecting medication errors1,2, improving quality of care by assessing timeliness of treatments3,4 and procedures5,6,productivity7,8 and efficiency9,10, integrating health information technologies and data management platforms11,12,and optimizing clinicians workload and time allocation13,14. Clinical workflow is commonly studied using qualitativeapproaches (ethnographic studies and interviews15), while quantitative methods use variations of time motionstudies16,17. Time motion studies (TMS) were originally developed in industrial engineering and focused on theanalysis of movements in a task with an emphasis on the amount of time required to perform the task18. Severaldifferent techniques have been compared for collecting quantitative workflow data19,20,21 (external observers, self reports or databases analysis; work sampling vs. continuous observation), defining the use of an external observer asthe gold standard to quantify clinical workflow22,23. It is a common practice among workflow researchers to use a combination of both qualitative andquantitative methods. Typically, researchers interpret and model qualitative data onto theoretical activity diagrams24,and then enrich the abstracted workflow with timedata25 by having observers record the duration of expectedordered milestones26. For example, a common practice in medication administration studies is to develop anidealized medication administration flowchart, and then have observers time the duration of each predicted step27,28.However, that practice does not take into account interruptions and the in