Development and validation of the predictive risk of death model for adult patients admitted to intensive care units in Japan: an approach to improve the accuracy of healthcare quality measures

Journal of Intensive Care

Tomoyuki Nakamura¹  Shunsuke Takaki²  Yoshitaka Aoki³  Satoru Hashimoto⁴  Hiromasa Irie⁵  Hiroyuki Ohbe⁶  Junji Kumasawa⁷  Hiroshi Okamoto⁸  Masatoshi Uchida⁹  Junji Hatakeyama^1,10  Takashi Tagami^1,11  Katsura Hayakawa^1,12  Nao Ichihara^1,13  Hiroaki Miyata^1,14  Hideki Endo^1,14  Hidenobu Shigemitsu^1,15  Kohei Takimoto^1,16  Hiroshi Kurosawa^1,17  Tatsuya Kawasaki^1,18  Eiji Hashiba^1,19  Shigehiko Uchino^2,20 

[1] Department of Anesthesiology and Critical Care Medicine, Fujita Health University School of Medicine, 1-98 Dengakugakubo, Kutsukake-cho, 470-1192, Toyoake, Aichi, Japan;Department of Anesthesiology and Critical Care Medicine, Yokohama City University, 3-9 Fukuura, Kanazawa-ku, 236-0004, Yokohama, Kanagawa, Japan;Department of Anesthesiology and Intensive Care Medicine, Hamamatsu University School of Medicine, 1-20-1 Handayama, Higashi-ku, Hamamatsu, 431-3192, Shizuoka, Japan;Department of Anesthesiology and Intensive Care Medicine, Kyoto Prefectural University of Medicine, 465 Kajii-cho, Kawaramachi-Hirokoji, Kamigyo-ku, 602-8566, Kyoto, Japan;Department of Anesthesiology, Kurashiki Central Hospital, 1-1-1 Miwa, Kurashiki, 710-8602, Okayama, Japan;Department of Clinical Epidemiology and Health Economics, School of Public Health, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-0033, Tokyo, Japan;Department of Critical Care Medicine, Sakai City Medical Center, 1-1-1 Ebaraji-cho, Nishi-ku, Sakai, 593-8304, Osaka, Japan;Department of Critical Care Medicine, St. Luke’s International Hospital, 9-1 Akashi-cho, Chuo-ku, 104-8560, Tokyo, Japan;Department of Emergency and Critical Care Medicine, Dokkyo Medical University, 880 Kitakobayashi, Mibu-machi, Shimotsuga-gun, 321-0293, Tochigi, Japan;Department of Emergency and Critical Care Medicine, National Hospital Organization Tokyo Medical Center, 2-5-1, Higashigaoka, Meguro-ku, 152-8902, Tokyo, Japan;Department of Emergency and Critical Care Medicine, Nippon Medical School Musashikosugi Hospital, 1-396 Kosugimachi, Nakahara-ku, 211-8533, Kawasaki, Kanagawa, Japan;Department of Emergency and Critical Care Medicine, Saitama Red Cross Hospital, 1-5 Shintoshin, Chuo-ku, 330-8553, Saitama, Japan;Department of Healthcare Quality Assessment, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-8655, Tokyo, Japan;Department of Healthcare Quality Assessment, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, 113-8655, Tokyo, Japan;Department of Health Policy and Management, School of Medicine, Keio University, 35 Shinanomachi, Shinjuku-ku, 160-8582, Tokyo, Japan;Department of Intensive Care Medicine, Graduate School of Medicine, Tokyo Medical and Dental University, 1-5-45 Yushima, Bunkyo-ku, 113-8519, Tokyo, Japan;Department of Intensive Care Medicine, Kameda Medical Center, 929 Higashi-cho, Kamogawa, 296-8602, Chiba, Japan;Department of Pediatric Critical Care Medicine, Hyogo Prefectural Kobe Children’s Hospital, 1-6-7 Minatojima Minamimachi, Chuo-ku, 650-0047, Kobe, Hyogo, Japan;Department of Pediatric Critical Care, Shizuoka Children’s Hospital, 860 Urushiyama, Aoi-ku, 420-8660, Shizuoka, Shizuoka, Japan;Division of Intensive Care, Hirosaki University Hospital, 53 Honcho, Hirosaki, 036-8203, Aomori, Japan;Intensive Care Unit, The Jikei University School of Medicine, 3-19-18 Nishi-Shinbashi, Minato-ku, 105-8471, Tokyo, Japan;

关键词: Risk of death;    Risk prediction model;    Recalibration;    Benchmarking;    Quality improvement;    Quality indicator;   

DOI  :  10.1186/s40560-021-00533-z

来源: Springer

【 摘 要 】

BackgroundThe Acute Physiology and Chronic Health Evaluation (APACHE) III-j model is widely used to predict mortality in Japanese intensive care units (ICUs). Although the model’s discrimination is excellent, its calibration is poor. APACHE III-j overestimates the risk of death, making its evaluation of healthcare quality inaccurate. This study aimed to improve the calibration of the model and develop a Japan Risk of Death (JROD) model for benchmarking purposes.MethodsA retrospective analysis was conducted using a national clinical registry of ICU patients in Japan. Adult patients admitted to an ICU between April 1, 2018, and March 31, 2019, were included. The APACHE III-j model was recalibrated with the following models: Model 1, predicting mortality with an offset variable for the linear predictor of the APACHE III-j model using a generalized linear model; model 2, predicting mortality with the linear predictor of the APACHE III-j model using a generalized linear model; and model 3, predicting mortality with the linear predictor of the APACHE III-j model using a hierarchical generalized additive model. Model performance was assessed with the area under the receiver operating characteristic curve (AUROC), the Brier score, and the modified Hosmer–Lemeshow test. To confirm model applicability to evaluating quality of care, funnel plots of the standardized mortality ratio and exponentially weighted moving average (EWMA) charts for mortality were drawn.ResultsIn total, 33,557 patients from 44 ICUs were included in the study population. ICU mortality was 3.8%, and hospital mortality was 8.1%. The AUROC, Brier score, and modified Hosmer–Lemeshow p value of the original model and models 1, 2, and 3 were 0.915, 0.062, and < .001; 0.915, 0.047, and < .001; 0.915, 0.047, and .002; and 0.917, 0.047, and .84, respectively. Except for model 3, the funnel plots showed overdispersion. The validity of the EWMA charts for the recalibrated models was determined by visual inspection.ConclusionsModel 3 showed good performance and can be adopted as the JROD model for monitoring quality of care in an ICU, although further investigation of the clinical validity of outlier detection is required. This update method may also be useful in other settings.

【 授权许可】

CC BY