The National Early Warning Score (NEWS) is now over 15 years old and has been validated across a number of conditions and in different healthcare settings: NEWS has been used to predict outcomes with COPD
1
, after surgery 2
, in frail elderly patients 3
, pancreatitis 4
and many other conditions often by adapting it or comparing it to other scoring systems.With the advent of COVID-19 there have been attempts to design scores that capture prognosis of this specific patient group. In April a group from Wuhan compared NEWS with a range of other scoring tools used in Emergency Care
5
. In July an international group published a review of no less than 50 prognostic models for patients with COVID-19 that had been published at this point in time - Liu F.-Y.
- Sun X.-L.
- Zhang Y.
- Ge L.
- Wang J.
- Liang X.
- et al.
Evaluation of the Risk Prediction Tools for Patients with Coronavirus Disease 2019 in Wuhan, China: a Single-Centered, Retrospective, Observational Study.
SSRN Electron J. 2020; https://doi.org/10.2139/ssrn.3562483
6
. This has not stopped the drive to create yet more risk prediction models matched to specific settings: For patients seen in primary care QCOVID Wynants L., Van Calster B., Collins G.S., Riley R.D., Heinze G., Schuit E., et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal WHAT IS ALREADY KNOWN ON THIS TOPIC n.d.;7:18. https://doi.org/10.1136/bmj.m1328.
7
was suggested and for patients admitted to hospital the 4C score was promoted - Hippisley-Cox J.
- Coupland C.
Predicting risk of emergency admission to hospital using primary care data: derivation and validation of QAdmissions score.
BMJ Open. 2013; 3e003482https://doi.org/10.1136/bmjopen-2013-003482
8
. Only in November the BMJ published a suggestion for a scoring model for remote monitoring comprising no less than 11 parameters that were derived from a modified Delphi process 9
.- Greenhalgh T.
- Thompson P.
- Weiringa S.
- Neves A.L.
- Husain L.
- Dunlop M.
- et al.
What items should be included in an early warning score for remote assessment of suspected COVID-19? qualitative and Delphi study.
BMJ Open. 2020; 10e042626https://doi.org/10.1136/bmjopen-2020-042626
The primary role in any predictive test is its ability to discriminate between those patients that have a poor trajectory and those that will have a good trajectory to allow targeted allocation of resources. No test is perfect, but how good does a test have to be to be good enough?
In this issue, Kostakis et al. validate the use of the NEWS score in COVID patients using a Receiver Operator Characteristic (or ROC) curve, with the number of false positives plotted on the x axis and the number of true positives on the y axis
10
. The higher the arc of the curve, the greater the area under the curve (AUROC) and the closer the test gets to a perfect result – represented by a value of 1. A value of 0.5 represents no discrimination between groups. A value of 0.8 – 0.9 is considered an excellent result and higher than 0.9 outstanding. This does not mean however that the test is perfect, and there are still some patients that can slip through the net. In patients with COVID-19, Kostakis et al. found an AUROC value 0.882. This is similar to values found in other studies of predictive models performed in a variety of critically ill patients: for example Pimental et al. found values of 0.86 for NEWS2 and 0.881 for NEWS to predict in-hospital mortality in 48,898 patients at risk for type 2 respiratory failure 11
.The present study allows not only to evaluate the performance of NEWS for potential triage of patients but importantly compares it to performance in patients not affected by COVID-19. The AUROC vales for patients admitted during the same time period showed that in COVID-19 not detected patients the value was 0.882 and in COVID-19 not tested group the value was 0.875 This is a similar performance to not only their own historical controls, but also other studies on NEWS scores performed in other centres
11
, 12
. This adds weight to the results from this single-centre study and corroborates the use of NEWS as an early warning score for these patients.Which leads to what is probably the most important property of NEWS: it is widely applied and understood and can therefore be used for staff from different organisations or professional groups to communicate risk in an unequivocal way.
While clinicians and statisticians are excited about the mathematical properties of the various scoring model user experience and actual impact on clinical outcomes is all too often neglected. While the development of new scoring models for COVID might be useful for risk stratification in interventional research studies or comparing physiological patterns in epidemiological work for clinicians simplicity is probably at least as important.
In this context the paper by Kostakis and co-workers tell us probably just about enough: patients with COVID-19 still follow the same patterns of acute physiology where more abnormalities translate to worse outcomes. Outcomes in patients with COVID-19 might be somewhat worse than those in patients without COVID-19 but this is not dissimilar to the notion that patients with high NEWS scores who are frail and elderly or suffering with multiple co-morbidities will be at higher risk than those who are young and have never been ill before. Additional complexity increases cognitive load and by using a single scoring model across all diagnostic categories clinicians require minimal adjustment of their mental model.
Extrapolating use of any score to new populations for which the tool has not yet been validated is not without risk, and the sudden explosion of sick patients with COVID-19 has meant that determination of which patients need escalating care has been a work in progress. This work is now coming to fruition, and the validation of a familiar tool to fit seamlessly with our current practice is a welcome development.
Declaration of interests
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests:
Chris Subbe has worked with several of the authors as an expert reviewer for a grant that they received from the Wellcome Trust and as chair of the study board supporting their NIHR grants for the FOBS study.
References
- Why Methodology Is Important: Coffee as a Candidate Treatment for COVID-19?.J Clin Med. 2020; 9: 3691https://doi.org/10.3390/jcm9113691
- Dynamic individual vital sign trajectory early warning score (DyniEWS) versus snapshot national early warning score (NEWS) for predicting postoperative deterioration.Resuscitation. 2021; 157: 176-184https://doi.org/10.1016/j.resuscitation.2020.10.037
- Comparison of the National Early Warning Score (NEWS) and the Modified Early Warning Score (MEWS) for predicting admission and in-hospital mortality in elderly patients in the pre-hospital setting and in the emergency department.PeerJ. 2019; 7: e6947https://doi.org/10.7717/peerj.6947
- Development of the National Early Warning Score-Calcium Model for Predicting Adverse Outcomes in Patients with Acute Pancreatitis.J Emerg Nurs. 2020; 46: 171-179https://doi.org/10.1016/j.jen.2019.11.003
- Evaluation of the Risk Prediction Tools for Patients with Coronavirus Disease 2019 in Wuhan, China: a Single-Centered, Retrospective, Observational Study.SSRN Electron J. 2020; https://doi.org/10.2139/ssrn.3562483
Wynants L., Van Calster B., Collins G.S., Riley R.D., Heinze G., Schuit E., et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal WHAT IS ALREADY KNOWN ON THIS TOPIC n.d.;7:18. https://doi.org/10.1136/bmj.m1328.
- Predicting risk of emergency admission to hospital using primary care data: derivation and validation of QAdmissions score.BMJ Open. 2013; 3e003482https://doi.org/10.1136/bmjopen-2013-003482
- Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO Clinical Characterisation Protocol: development and validation of the 4C Mortality Score.BMJ. 2020; 370: 22https://doi.org/10.1136/bmj.m3339
- What items should be included in an early warning score for remote assessment of suspected COVID-19? qualitative and Delphi study.BMJ Open. 2020; 10e042626https://doi.org/10.1136/bmjopen-2020-042626
- The performance of the National Early Warning Score and National Early Warning Score 2 in hospitalised patients infected by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).Resuscitation. 2021; 159: 150-157https://doi.org/10.1016/j.resuscitation.2020.10.039
- A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: A multi-centre database study.Resuscitation. 2019; 134: 147-156https://doi.org/10.1016/j.resuscitation.2018.09.026
- Decision-tree early warning score (DTEWS) validates the design of the National Early Warning Score (NEWS).Resuscitation. 2014; 85: 418-423https://doi.org/10.1016/j.resuscitation.2013.12.011
Article info
Publication history
Published online: December 18, 2020
Identification
Copyright
© 2020 Elsevier B.V. All rights reserved.
