Reliability of team-based self-monitoring in critical events: a pilot study

BMC Emergency Medicine

Research Article

Suzan Kakat¹  Mary Lane²  Lynda Menadue³  Winston Banya⁴  Martin Stocker⁵  Margarita Burmester⁶  Kumi De Costa⁶  Ajay Desai⁶  Julie Combes⁶ 

[1] CICU Great Ormond Street Hospital, Great Ormond Street, WC1N 3JH, London, UK;Department of Anaesthesia, Royal Brompton Hospital, Sydney Street, SW3 6NP, London, UK;Department of Anaesthesia, Singapore General Hospital, Outram Road, 169608, Singapore, Singapore;Epidemiology and Biostatistics, Royal Brompton Hospital, Sydney Street, SW3 6NP, London, UK;NICU/PICU Children’s Hospital Lucerne, CH-6000, Lucerne 16, Switzerland;PICU Royal Brompton Hospital, Sydney Street, SW3 6NP, London, UK;

关键词: Teamwork;    Self-assessment;    Mayo high performance teamwork scale;    Resuscitation;    Patient safety;   

DOI  :  10.1186/1471-227X-13-22

 received in 2013-06-01, accepted in 2013-11-26,  发布年份 2013

来源: Springer

【 摘 要 】

BackgroundTeamwork is a critical component during critical events. Assessment is mandatory for remediation and to target training programmes for observed performance gaps.MethodsThe primary purpose was to test the feasibility of team-based self-monitoring of crisis resource management with a validated teamwork assessment tool. A secondary purpose was to assess item-specific reliability and content validity in order to develop a modified context-optimised assessment tool.We conducted a prospective, single-centre study to assess team-based self-monitoring of teamwork after in-situ inter-professional simulated critical events by comparison with an assessment by observers. The Mayo High Performance Teamwork Scale (MHPTS) was used as the assessment tool with evaluation of internal consistency, item-specific consensus estimates for agreement between participating teams and observers, and content validity.Results105 participants and 58 observers completed the MHPTS after a total of 16 simulated critical events over 8 months. Summative internal consistency of the MHPTS calculated as Cronbach’s alpha was acceptable with 0.712 for observers and 0.710 for participants. Overall consensus estimates for dichotomous data (agreement/non-agreement) was 0.62 (Cohen’s kappa; IQ-range 0.31-0.87). 6/16 items had excellent (kappa > 0.8) and 3/16 good reliability (kappa > 0.6). Short questions concerning easy to observe behaviours were more likely to be reliable. The MHPTS was modified using a threshold for good reliability of kappa > 0.6. The result is a 9 item self-assessment tool (TeamMonitor) with a calculated median kappa of 0.86 (IQ-range: 0.67-1.0) and good content validity.ConclusionsTeam-based self-monitoring with the MHPTS to assess team performance during simulated critical events is feasible. A context-based modification of the tool is achievable with good internal consistency and content validity. Further studies are needed to investigate if team-based self-monitoring may be used as part of a programme of assessment to target training programmes for observed performance gaps.

【 授权许可】

CC BY   
© Stocker et al.; licensee BioMed Central Ltd. 2013