Using Markov chain model to evaluate medical students’ trajectory on progress tests and predict USMLE step 1 scores---a retrospective cohort study in one medical school

BMC Medical Education

Ling Wang¹  Heather S. Laird-Fick¹  David Solomon²  Carol J. Parker³ 

[1] Department of Medicine, Michigan State University, 909 Wilson Rd, 120 West Fee Hall, 48824, East Lansing, MI, USA;Department of Medicine, Michigan State University, 909 Wilson Rd, 120 West Fee Hall, 48824, East Lansing, MI, USA;Office of Medical Education Research and Development (OMERAD), Michigan State University, East Lansing, MI, USA;Office of Medical Education Research and Development (OMERAD), Michigan State University, East Lansing, MI, USA;

关键词: Longitudinal study;    Markov chain model;    Progress tests;    USMLE step 1 performance;   

DOI  :  10.1186/s12909-021-02633-8

来源: Springer

【 摘 要 】

BackgroundMedical students must meet curricular expectations and pass national licensing examinations to become physicians. However, no previous studies explicitly modeled stages of medical students acquiring basic science knowledge. In this study, we employed an innovative statistical model to characterize students’ growth using progress testing results over time and predict licensing examination performance.MethodsAll students matriculated from 2016 to 2017 in our medical school with USMLE Step 1 test scores were included in this retrospective cohort study (N = 358). Markov chain method was employed to: 1) identify latent states of acquiring scientific knowledge based on progress tests and 2) estimate students’ transition probabilities between states. The primary outcome of this study, United States Medical Licensing Examination (USMLE) Step 1 performance, were predicted based on students’ estimated probabilities in each latent state identified by Markov chain model.ResultsFour latent states were identified based on students’ progress test results: Novice, Advanced Beginner I, Advanced Beginner II and Competent States. At the end of the first year, students predicted to remain in the Novice state had lower mean Step 1 scores compared to those in the Competent state (209, SD = 14.8 versus 255, SD = 10.8 respectively) and had more first attempt failures (11.5% versus 0%). On regression analysis, it is found that at the end of the first year, if there was 10% higher chance staying in Novice State, Step 1 scores will be predicted 2.0 points lower (95% CI: 0.85–2.81 with P < .01); while 10% higher chance in Competent State, Step 1scores will be predicted 4.3 points higher (95% CI: 2.92–5.19 with P < .01). Similar findings were also found at the end of second year medical school.ConclusionsUsing the Markov chain model to analyze longitudinal progress test performance offers a flexible and effective estimation method to identify students’ transitions across latent stages for acquiring scientific knowledge. The results can help identify students who are at-risk for licensing examination failure and may benefit from targeted academic support.

【 授权许可】

CC BY