Advertisement

Predicting unexpected deterioration of high-risk hospitalized patients during the COVID-19 pandemic: A multicenter cohort study

  • Sang Hyuk Kim
    Affiliations
    Division of Pulmonology and Critical Care Medicine, Samsung Medical Center, Department of Medicine, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
    Search for articles by this author
  • Hayoung Choi
    Affiliations
    Division of Pulmonary, Allergy, and Critical Care Medicine, Department of Internal Medicine, Hallym University Kangnam Sacred Heart Hospital, Seoul, Republic of Korea
    Search for articles by this author
  • Hyun Lee
    Affiliations
    Division of Pulmonary Medicine and Allergy, Department of Internal Medicine, Hanyang University College of Medicine, Seoul, Republic of Korea
    Search for articles by this author
  • Ji Young Hong
    Affiliations
    Division of Pulmonary, Allergy and Critical Care Medicine, Department of Internal Medicine, Hallym University Chuncheon Sacred Heart Hospital, Chuncheon, Republic of Korea
    Search for articles by this author
  • Youlim Kim
    Correspondence
    Corresponding author.
    Affiliations
    Division of Pulmonary, Allergy and Critical Care Medicine, Department of Internal Medicine, Hallym University Chuncheon Sacred Heart Hospital, Chuncheon, Republic of Korea
    Search for articles by this author
      To the Editor,
      Due to the coronavirus 19 (COVID-19) outbreak, there have been major healthcare utilization changes among patients with chronic diseases.
      • Malhotra C.
      • Chaudhry I.
      • Ozdemir S.
      • Finkelstein E.A.
      Reduced health-care utilization among people with chronic medical conditions during coronavirus disease.
      Therefore, medical resources should be redistributed so that medical staff can appropriately manage COVID-19 patients.
      • Ahn S.
      • Kim S.
      • Koh K.
      Changes in healthcare utilization, spending, and perceived health during COVID-19: a longitudinal study from Singapore.
      Along with healthcare environment changes, assessment of risk of deterioration among high-risk patients also needs to change.
      The National Early Warning Score (NEWS) is a scoring system that can predict hospitalized patient deterioration.
      • Smith G.B.
      • Prytherch D.R.
      • Meredith P.
      • Schmidt P.E.
      • Featherstone P.I.
      The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.
      Since NEWS threshold is low, many patients could be classified as high-risk by NEWS during COVID-19 outbreak.
      • Petersen J.A.
      Early warning score-challenges and opportunities in the care of deteriorating in-hospital patients.
      Depending on the scale of the COVID-19 outbreak, intensive care prioritization may be required in a limited intensive care unit (ICU).
      • Proudfoot A.G.
      • O’Brien B.
      • Schilling R.
      • Gould D.W.
      • McGlennan A.
      Rapid establishment of a COVID-19 critical care unit in a convention centre: the Nightingale Hospital London experience.
      We hypothesized that unexpected deterioration among high-risk patients could be predicted using basic clinical information added to NEWS parameters using a machine learning method.
      We conducted a multicenter-based, retrospective, consecutive cohort study at five hospitals associated with Hallym Medical Center. The study period was from February 2020 to December 2020, including three COVID-19 pandemic periods in Korea (February, August, and December). High-risk patients were defined as patients with a National Early Warning Score (NEWS) over 7. Outcomes included unexpected ICU admission, which was defined as transfer to the ICU within 24 h after being classified as high-risk, and in-hospital mortality.
      The random forest method was used to predict outcomes with 500 trees using the ‘randomForest’ package of R, and the accuracy of the developed model was compared with the NEWS. During development, age, sex, body mass index, comorbidities, and unscored NEWS parameters were included. Training and test sets were assigned at a 7:3 ratio. Comorbidities consisted of cardiovascular, pulmonary, gastrointestinal, neurological, genitourinary disease, and cancer based on the 10th International Classification of Disease code. All statistical analyses were performed using R 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).
      A total of 6576 NEWS was recorded from 2118 patients. The median (interquartile range) of age, body mass index, and NEWS were 73 (64–82) years, 21 (18–24), and 8 (7–9), respectively. The proportion of male, cardiovascular, pulmonary, gastrointestinal, neurological, genitourinary disease, and cancer patients was 60.7%, 4.6%, 47.7%, 9.1%, 2.6%, 4.4%, 28.8%, respectively. The random forest model showed a good predictive ability for ICU admission (area under the receiver operating curve [AUROC] 0.787, 95% confidence interval [CI] 0.748–0.825) and excellent predictive ability for in-hospital mortality (AUROC 0.881 95% CI 0.866–0.895, which were higher than that of NEWS (p < 0.001) (Fig. 1).
      Fig. 1
      Fig. 1Receiver operating characteristic curve for unexpected intensive care unit admission and in-hospital mortality using the predictive model. The 95% CI and p-value were calculated using 2000 stratified bootstrap replicates. ICU, intensive care unit; NEWS, National Early Warning Score; AUROC, area under the receiver operating curve; CI, confidence interval.
      Our study found that a random forest-based model had a good predictive ability for unexpected ICU admission and in-hospital mortality in high-risk patients. Decision-making for prioritizing intensive care among acute illness patients would be facilitated by our model. Also, our model used immediately measurable parameters of hospitalized patients. The algorithm was also intuitive and straightforward. Therefore, we believe that medical institutions with limited resources can prioritize intensive care using this method.

      Ethics and patient consent

      This study was conducted following the Helsinki declaration and was approved by the Institutional Review Board (IRB) of Hallym University Chuncheon Sacred Heart Hospital (IRB number: 2021-02-007). The need for written informed consent was waived because this was a retrospective cohort study.

      Authors’ contributions

      Conceptualization, Formal analysis and Writing-original draft: S.H.K., Review and editing: H.C., H.L. and J.Y.H, Supervision and validation: Y.K. All authors read and approved the final manuscript.

      Data availability

      The data of our study can be made fully available when the manuscript is accepted for publication.

      Funding sources

      None.

      Conflicts of interests

      The authors do not have any competing interests to declare.

      References

        • Malhotra C.
        • Chaudhry I.
        • Ozdemir S.
        • Finkelstein E.A.
        Reduced health-care utilization among people with chronic medical conditions during coronavirus disease.
        Proc Singapore Healthc. 2019; 2020 (2010105820964533)
        • Ahn S.
        • Kim S.
        • Koh K.
        Changes in healthcare utilization, spending, and perceived health during COVID-19: a longitudinal study from Singapore.
        Spending, and perceived health during COVID-19: a longitudinal study from Singapore (August 7, 2020).2020
        • Smith G.B.
        • Prytherch D.R.
        • Meredith P.
        • Schmidt P.E.
        • Featherstone P.I.
        The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death.
        Resuscitation. 2013; 84: 465-470
        • Petersen J.A.
        Early warning score-challenges and opportunities in the care of deteriorating in-hospital patients.
        University of Copenhagen, Faculty of Health and Medical Sciences, 2016 (Phd Thesis)
        • Proudfoot A.G.
        • O’Brien B.
        • Schilling R.
        • Gould D.W.
        • McGlennan A.
        Rapid establishment of a COVID-19 critical care unit in a convention centre: the Nightingale Hospital London experience.
        Intensive Care Med. 2021; 47: 349-351