Article Text

Predictive value of the National Early Warning Score 2 for hospitalised patients with viral respiratory illness is improved by the addition of inspired oxygen fraction as a weighted variable
  1. Jonathan Clarke1,
  2. Jack Gallifant2,
  3. David Grant3,
  4. Nishita Desai1 and
  5. Guy Glover1
  1. 1Department of Critical Care, Guy’s and St Thomas’ NHS Foundation Trust, London, UK
  2. 2Department of Critical Care, Imperial College Healthcare NHS Trust, London, UK
  3. 3Department of Medical Informatics, Guy’s and St Thomas’ NHS Foundation Trust, London, UK
  1. Correspondence to Dr Guy Glover; guy.glover{at}gstt.nhs.uk

Abstract

Objectives The National Early Warning Score 2 (NEWS2) is validated for predicting acute deterioration, however, the binary grading of inspired oxygen fraction (FiO2) may limit performance. We evaluated the incorporation of FiO2 as a weighted categorical variable on NEWS2 prediction of patient deterioration.

Setting Two hospitals at a single medical centre, Guy’s and St Thomas’ NHS Foundation Trust.

Design Retrospective cohort of all ward admissions, with a viral respiratory infection (SARS-CoV-2/influenza).

Participants 3704 adult ward admissions were analysed between 01 January 2017 and 31 December 2021.

Methods The NEWS-FiO2 score transformed FiO2 into a weighted categorical variable, from 0 to 3 points, substituting the original 0/2 points. The primary outcome was a composite of cardiac arrest, unplanned critical care admission or death within 24 hours of the observation. Sensitivity, positive predictive value (PPV), number needed to evaluate (NNE) and area under the receiver operating characteristic curve (AUROC) were calculated. Failure analysis for the time from trigger to outcome was compared by log-rank test.

Results The mean age was 60.4±19.4 years, 52.6% were men, with a median Charlson Comorbidity of 0 (IQR 3). The primary outcome occurred in 493 (13.3%) patients, and the weighted FiO2 score was strongly associated with the outcome (p=<0.001). In patients receiving supplemental oxygen, 78.5% of scores were reclassified correctly and the AUROC was 0.81 (95% CI 0.81 to 0.81) for NEWS-FiO2 versus 0.77 (95% CI 0.77 to 0.77) for NEWS2. This improvement persisted in the whole cohort with a significantly higher failure rate for NEWS-FiO2 (p=<0.001). At the 5-point threshold, the PPV increased by 22.0% (NNE 6.7) for only a 3.9% decrease in sensitivity.

Conclusion Transforming FiO2 into a weighted categorical variable improved NEWS2 prediction for patient deterioration, significantly improving the PPV. Prospective external validation is required before institutional implementation.

  • COVID-19
  • Critical Care

Data availability statement

Data are available upon reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

  • Previous work has demonstrated the significant limitations in the specificity of the National Early Warning Score 2 (NEWS2) score and the suitability of NEWS-inspired oxygen fraction (FiO2) in patients receiving supplemental oxygen. However, this tool has not been generalised to all patients, which is crucial if this score is to be translated into clinical practice.

WHAT THIS STUDY ADDS

  • In a cohort of over 3700 patients, we demonstrate the benefit of transforming FiO2 from a binary to a weighted categorical variable. The NEWS-FiO2 better stratifies patient risk of deterioration and offers a clinically meaningful improvement in positive predictive value and number needed to evaluate, corresponding to roughly two fewer alarms per day, with a limited decrease in sensitivity.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

  • This improvement provides evidence that NEWS-FiO2 could be clinically operationalised using already collected clinical data. Our results have national and international importance in fulfilling the Royal College of Physicians’ call for further research on this topic and laying the foundations for the introduction of NEWS-FiO2 into clinical practice.

Introduction

Health agencies worldwide have advocated the benefits of early identification of the deteriorating patient for over two decades.1 2 Similarly, calls for an in-hospital ‘Chain of Prevention’ model, mirroring that employed by the Resuscitation Council, have led to the adoption of both early warning scores (EWS) and rapid response systems (RRS) in hospitals.1–4 The National Early Warning Score 2 (NEWS2) is in widespread use throughout the UK and internationally as the second iteration of a physiological aggregate weighted track and trigger scoring system using six simple parameters: respiration rate, oxygen saturation, systolic blood pressure, pulse rate, level of consciousness (including new confusion) and temperature.5 Each parameter is attributed a score based on the degree of variation from ‘normal ranges’, and the total score is augmented by two points if supplemental oxygen is required. The aggregate score is then used to trigger specific responses. A threshold of five points is defined as the point at which an urgent review by a clinician trained in managing the acutely ill patient is required. A score of 7 or more should prompt an emergency review by a team with critical care competencies.

NEWS2 has been validated for the prediction of, and clinical response to, the deteriorating patient.6

However, the current binary grading of inspired oxygen fraction (FiO2) may limit the predictive power of NEWS2 by failing to provide additional weight as the patient deteriorates with rising oxygen requirements. This has been particularly pronounced during the COVID-19 pandemic, where patients frequently maintained stable parameters despite a marked increase in their FiO2 requirements.7 8 We have previously demonstrated that FiO2 as a continuous variable had greater predictive validity than binary grading in patients with COVID-19.9 This limitation has been acknowledged by the Royal College of Physicians, who subsequently issued additional guidance and called for further research into the impact of FiO2 on NEWS2 performance.10 In the 24 hours prior to an unplanned intensive care unit (ICU) admission, up to 75% of patients have been shown to have a new or increasing oxygen requirement.11 Improving discrimination in oxygen administration could offer greater stratification of patient risk.

This failure to separate patients is problematic in patients with low supplemental oxygen requirements, for example, during the management of small deficits in gas exchange (eg, postoperative atelectasis) or as part of ‘routine patient care’, for example, during concurrent patient-controlled analgesia systems. These patients score the same two additional points as those receiving high FiO2 via non-rebreather masks. This may increase the patient’s overall NEWS2 score disproportionately compared with their true risk of an adverse event. Furthermore, this may contribute to one of the key limitations of NEWS2, namely the low positive predictive value12 which is a function of the low prevalence of the outcome. This feature increases the rate of ‘false alerts’ and may contribute to alarm fatigue in clinical practice.13

One previous study demonstrated that a NEWS-FiO2 model is associated with improved performance14 but the FiO2 thresholds developed have yet to be externally validated. This study addresses this gap by evaluating whether the incorporation of FiO2 transformed into an additional weighed parameter in a NEWS-FiO2 model improved the predictive validity in a cohort of patients with viral respiratory illness (COVID-19 and influenza).

Methods

A retrospective observational cohort study was performed in two hospitals at a single medical centre (Guy’s and St Thomas’ NHS Foundation Trust, (GSTT)) between 01 January 2017 and 31 December 2021. The hospitals employ an electronic health record (EHR) system for recording physiological observations in general inpatient wards, with NEWS fully embedded in practice since 2012 and updated to NEWS2 in 2017. NEWS2 includes a modified peripheral oxygen saturation (SpO2) scale for patients with chronic hypercapnic respiratory failure, formalising a modification that had been introduced with NEWS as a local exception in our system (the so called Chronic Respiratory Early Warning Score (CREWS)).15 Observation frequency and escalation were protocolised in accordance with NEWS2 guidance. There is a dedicated 24/7 critical care outreach team (CCOT) staffed by advanced nurse practitioners with dedicated middle and senior-grade critical care doctors which is triggered by elevated NEWS2 as per national guidance. High flow nasal oxygen use is limited to specific wards with the support of the CCOT, and was not used outside of critical care in the COVID-19 period due to concerns about efficacy, oxygen resilience and infection control. Continuous positive airway pressure is not routinely used outside of critical care and therefore is a criterion for critical care admission.

The study cohort included all adult (≥17 years) patients admitted to general inpatient wards with a diagnosis of SARS-CoV-2 infection (COVID-19) or influenza, as two common causes of hypoxic respiratory failure. The COVID-19 cohort was defined by patients with a positive test by PCR from nasopharyngeal or throat swab, within 4 days of the index admission or in the 7 days prior to admission between 01 March 2020 and 30 March 2021. We used these dates to focus on patients in the first two waves of the COVID-19 pandemic only, as thereafter there was increasing heterogeneity due to patients being diagnosed with COVID-19 which was not the primary diagnosis or was not a cause of hypoxaemic respiratory failure. We excluded positive tests after 4 days to avoid confounding by patients who may have had a nosocomial acquisition and therefore for whom COVID-19 may not be the primary cause of illness. In addition to a COVID-19 cohort we included an influenza cohort, in recognition that the pathophysiology of COVID-19 may be novel and to test our hypothesis in another common cause of hypoxaemic respiratory failure. For the diagnosis of influenza, we included all adult (≥17 years) patients admitted to a general inpatient ward, with a discharge International Classification of Diseases 10th Revision (ICD-10) code (any coding position) of influenza (J09, J10, J11 or J12)—(excluding J12.8 and 12.9 if admitted after 1/1/20 as this was found to have been applied for emergency use in COVID-19 during the early phase of the pandemic) from 01 January 2017 (the onset of the study database) to 30 December 2021 inclusive. After excluding J12.8 and J12.9 we did not identify any patient who fulfilled criteria for both COVID-19 and influenza cohorts.

The following criteria were applied for the exclusion of physiological observations: preceding the confirmation of COVID-19; erroneously attributed to have occurred after critical care admission (due to mistiming of administrative processes of ward transfer); occurring after the primary outcome event in either cohort; incomplete physiological observation set which hence could not generate a NEWS2 score.

The primary outcome was a composite of peri-arrest, cardiac arrest, unplanned critical care admission or death within 24 hours of the observation, in keeping with previous studies,16 with each EWS value treated as an individual predictor. Hence, risk was assessed continually throughout the patient’s length of stay. The time of the primary outcome was determined by the occurrence of the first event for patients who experienced more than one of the three individual outcomes. Accordingly, any subsequent events including critical care readmissions were not included. Peri-arrest or cardiac arrest were defined as a CCOT review coded as ‘2222 emergency’ (2222 being the generic telephone number for an emergency call in UK hospitals), and that were followed by a critical care admission or death within 24 hours—a definition that had been previously internally validated (data unpublished). The timing of critical care admission was defined by the time of the recording of the first heart rate to minimise errors associated with administrative processes.

Data is extracted from the GSTT Data Warehouse, which serves as an aggregate repository of data from multiple electronic sources, developed using the Health Catalyst Data Operating System (Health Catalyst, Salt Lake City, Utah, USA) and continuous since 01 January 2017. Variables included demographics, comorbidities, date and time of COVID-19 diagnosis, longitudinal physiological observations (respiratory rate, oxygen saturation, systolic blood pressure, pulse rate, level of consciousness, temperature and oxygen delivered), Charlson Comorbidity Index17 and the Dr Foster Global Frailty Score.18 Data on the use of the CREWS/scale 2 modification was not available, however, the NEWS score used already reflected the use of the modification in clinical practice.

FiO2 was transformed into a weighted categorical variable and assigned 0–3 points as previously published,14 substituting the original 0 points for no oxygen therapy or 2 points for those requiring any supplementary oxygen. In the proposed NEWS-FiO2 model, 0 points were attributed to those patients not requiring supplemental oxygen, 1 point for an FiO2 between 0.22 and 0.37, 2 points for an FiO2 between 0.371 and 0.53 and 3 points for an FiO2 of greater than 0.53. For fixed-performance oxygen devices, the set FiO2 value was used, while for variable-performance devices the FiO2 was estimated from the flow rate, as previously described19 (online supplemental table S1). The impact of an alternative method of estimating FiO2 was also tested, which attempts to account for the impact of the patient’s respiratory minute volume on ambient air entrainment and hence on the FiO2, referred to as the ‘Bateman formula’.20 This equation states that FiO2=(O2 flow rate+0.21 (minute volume–O2 flow rate))/minute volume. The minute volume was estimated using a fixed tidal volume (450 mL) multiplied by the recorded respiratory rate as previously published.14

Statistical methods

Data is presented as mean±SD, median (IQR) or number (%) and comparisons were performed using t-tests, Wilcoxon rank or χ2 tests as appropriate. The rate of the primary outcome, stratified by the value score of the weighted FiO2 parameter, was compared across groups with the χ2 test. The change in score from NEWS2 to NEWS-FiO2 is calculated, in addition to the change in classification of the primary outcome.

For NEWS2/NEWS-FiO2, the area under the receiving operator characteristic curves (AUROC) were calculated with 95% CIs. However, due to the limitations of using the AUROC to assess the efficacy of an EWS,21 we also report the positive predictive value (PPV) and numbers needed to evaluate (NNE) at each integer threshold of the EWS. The NNE is the number of patients who meet the threshold and who would need to be escalated to detect one true positive case, being the reciprocal of the PPV.21

Failure analysis was performed to compare time from meeting an EWS threshold to the occurrence of the primary outcome for the first 24 hours, making the following comparisons: (1) NEWS2 ≥5 versus NEWS-FiO2 ≥5 and (2) NEWS-FiO2<5 versus ≥5; time to event was compared using the log-rank test.

For all analyses, a threshold of p<0.05 was considered statistically significant. All analysis was performed in Stata (StataCorp 2013 Stata Statistical Software: Release 13). Strengthening the Reporting of Observational Studies in Epidemiology guidance for reporting cohort studies was followed (online supplemental file S2). Ethics approval was granted by the Research and Ethics Committee of GSTT (REC Number: 20/HRA/1871).

Patient and public involvement

Patients and/or the public did not take part in the development, conduct, reporting or dissemination of this study.

Results

Study population

After exclusions, 3704 patients were included in this analysis, of which 2468 (66.6%) were in the COVID-19 cohort and 1236 (33.3%) were in the influenza cohort (online supplemental figure 1). In the influenza cohort, 70% had the code in the first coding position and 91% in the first to third coding position, indicating that viral illness was the primary cause, or at least a significant contributor to the admission. 6020 (4.3%) of potential observations sets were excluded as they were incomplete and hence there was no associated NEWS2 score (4876 (4.1%) for patients without the primary outcome versus 1144 (6.0%) for patients with the primary outcome), hence the total number of observation sets analysed was 133 349. Baseline characteristics are shown in table 1, presented by the occurrence of the primary outcome.

Table 1

Baseline characteristics of the whole cohort, presented by the occurrence of the primary outcome

Of the 3704 patients included in this study, the median age on admission was 61.2 years with just over half being men. Over one-third of the patients were of black, Asian or mixed ethnicity and there was generally a low burden of comorbidity and frailty. Those who sustained the primary outcome event tended to be older, were more likely to be male and had higher Charlson Comorbidity and Global Frailty Scores (table 1). In addition, the observations in patients with the primary outcome were more abnormal, there was a greater proportion with supplemental oxygen and the EWS were numerically higher as compared with patients without the primary outcome.

Primary outcome

Overall, an adverse event occurred in 493 (13.3%) patients, of which 11 (2.2%) suffered a peri-arrest or cardiac arrest, 298 (60.4%) had an unplanned critical care admission and 184 (37.3%) died on the ward as their first event. The primary outcome occurred following 3536 (2.65%) of the observation sets.

NEWS2 versus NEWS-FiO2

55 751 (41.8%) of the included data sets included supplemental oxygen and therefore scored two additional points on the NEWS2. Applying the NEWS-FiO2 model to those patients with supplementary oxygen, led to a change in score follows: in 12.3% there was no change in the total score, in 78.5% the score decreased by one point and in 9.2% the score increased by one point. 43 743 (78.5%) EWS were reclassified correctly (42 514 (76.3%) decreased when the primary outcome did not occur; 1229 (2.2%) increased when the primary outcome did occur), as compared with 5133 (9.2%) EWS which were reclassified incorrectly (3890 (7.0%) increased when the primary outcome did not occur and 1243 (2.2%) decreased when the primary outcome did occur).

The weighted FiO2 score was strongly associated with the primary outcome (figure 1); namely, increased oxygen requirements were correlated with an increased crude risk of a primary outcome event (0, 0.4%, 1, 3.1%, 2, 15.9%, 3, 25.2%; (p=<0.001).

Figure 1

The rate of the primary outcome according to the value assigned to the FiO2 in the NEWS-FiO2 model. FiO2, fractional inspired concentration of oxygen; NEWS2, National Early Warning Score 2.

Compared with NEWS2 the AUROC was greater for NEWS-FiO2 (table 2). This improvement was particularly evident when the analysis was restricted to those patients receiving supplemental oxygen (table 3).

Table 2

Area under receiver operating characteristic curve with 95% CIs for the predictive validity for the primary outcome; all included observations (n=133 349)

Table 3

Area under receiver operating characteristic curve with 95% CIs for the predictive validity for the primary outcome; observations with supplemental oxygen only (n=55 751)

At the clinically and operationally relevant 5-point threshold, the specificity of NEWS-FiO2 was 89.4% versus 85.8% for NEWS2. The PPV increased by 22% from 12.2 for NEWS2 to 14.9 for NEWS-FiO2, and the NNE decreased from 8.2 to 6.7. This is balanced with a decrease of 3.9% in sensitivity, comparing NEWS2 to NEWS-FiO2 (online supplemental tables S2 & S3).

Estimating the FiO2 using the Bateman formula resulted in a decrease in the AUROC when compared with the predicted model (0.80 (95% CI 0.79 to 0.80) vs 0.81 (0.81 to 0.81)). At the 5-point threshold, the Bateman method resulted in a reduced PPV (12.8 vs 14.9), an increased NNE (7.8 vs 6.7) and a reduced specificity (86.6 vs 89.4) as compared with the predicted method.

Failure analysis

At the 5-point threshold, the failure rate was higher for patients with NEWS-FiO2 than with NEWS2, indicating that the novel score portends higher risk (log-rank test p=<0.001) (figure 2). Additionally, when analysing solely the NEWS-FiO2 scores, the ≥5 threshold shows discrimination, with the higher scores being associated with a markedly higher risk of the primary outcome in the following 24 hours. (Log-rank test p=<0.001) (figure 3).

Figure 2

Failure analysis—the proportion of observations followed by the primary outcome within 24 hours; all early warning score values ≥5, comparing NEWS2 (dashed line) and NEW-FiO2.(solid line) FiO2, fractional inspired oxygen concentration; NEWS, National Early Warning Score.

Figure 3

Failure analysis—the proportion of observations followed by the primary outcome within 24 hours; NEW-FiO2 <5 (dashed line) versus ≥5 (solid line). FiO2, fractional inspired oxygen concentration; NEWS, National Early Warning Score.

Discussion

In this cohort of patients admitted to a general inpatient ward with a diagnosis of either COVID-19 or influenza, 13.3% went on to experience a peri-arrest or cardiac arrest, unplanned critical care admission or death on the ward. This study demonstrates that the weighted FiO2 parameter score is strongly associated with outcome and that incorporating this into a NEWS-FiO2 model outperforms the existing NEWS2, with an increased PPV at limited cost to the overall sensitivity of the scoring system. This benefit holds true in two common viral respiratory illnesses. As expected, the effect is mediated through benefit in patients receiving supplemental oxygen, but is sufficiently robust to remain evident across the whole cohort, regardless of supplemental oxygen, in whom practically any novel EWS would be implemented. By changing from a pooled risk for all patients on oxygen regardless of FiO2 to a stratified risk based on the FiO2 requirement, the NEWS-FiO2 score addresses two key limitations of NEWS2—namely, lack of PPV and the fact that it is not sensitive to deterioration in the SpO2-FiO2 ratio. The fact that NEWS-FiO2 confers a reduction in score in over three quarters of patients on supplemental oxygen suggests an improved predictive value of the NEWS-FiO2 compared with NEWS2, primarily by minimising the score of those on low FiO2 supplemental oxygen who are at lower risk of deterioration to the primary outcome.

The findings of our study have the potential to bring clinically relevant benefits. Given that the majority of patients in hospitals on supplemental oxygen actually have low FiO2 requirements22 and the majority of these are at low risk of deterioration, NEWS-FiO2 appropriately decreased the EWS and, therefore the NNE. In our system, this translates to approximately one fewer referral of a patient meeting the 5-point threshold per 12-hour shift, thus increasing the efficiency of the RRS and reducing the likelihood of ‘alarm fatigue’.13 Equally, the increase in PPV means that the patient meeting the 5-point threshold for an urgent review, is more likely to genuinely warrant CCOT involvement as they are at a higher risk of deterioration, a fact supported by the higher failure rate after NEWS-FiO2 ≥5, as compared with NEWS2. As previously discussed, patients generally have an increase in oxygen requirement as an antecedent to deterioration, and it is this population who have the greatest potential to benefit from employing NEWS-FiO2.

Despite our findings, it is noted that the performance of NEWS2 in this study was similar to previous validation studies,23 justifying the literature that has validated and recommended the use of NEWS2 in COVID-19.24

We have also demonstrated that the method of estimating FiO2 makes a small but material difference to the performance of the scores. While the ‘Bateman formula’ attempts to account for the impact of the minute ventilation on the FiO2 and may give a more accurate representation of the true FiO2 for variable performance devices, it still relies on assumptions regarding the minute ventilation of the patient and would be significantly more challenging to operationalise without an EHR system to calculate this automatically. We opted to replicate a method previously reported, however acknowledge that the use of a fixed tidal volume is simplistic. While changing the tidal volume does inversely affect the estimated FiO2, data of Malycha et al14 demonstrates that the impact is likely to be limited. This component of our study demonstrates that the method of estimating FiO2 is important in any future work on NEWS-FiO2 and should be standardised.

The strengths of this study include the relatively large sample size as well as the large volume of observations included. The inclusion of a viral pneumonia other than COVID-19 is important as an indicator of validity in a wider population of common respiratory illnesses. This study builds on an already validated EWS which is well-adopted in clinical practice, using data routinely recorded at the bedside. The addition of FiO2 to the observations collected could be easily operationalised by minor modifications to any EHR system or a simple ‘look-up’ reference table for those recorded manually.

The limitations of our study are principally that it is a single-centre study, in an academic medical setting, performed in a retrospective fashion and patient inclusion was limited to those with a COVID-19 or influenza diagnosis. While this study adds external validity to the initial description of NEWS-FiO2, it is not known if the findings are further generalisable across different healthcare settings, patient demographics and primary diagnosis.25 This population had an uncommonly high rate of supplemental oxygen administration and thus, in a more general population, a statistically significant benefit may not be demonstrated across the whole cohort. Provided the burden of widespread implementation is limited, this ought not to discourage adoption for the benefits in the patients who do require supplemental oxygen. It is not known if the FiO2 thresholds that have been previously published and which we used are optimal, however, a simple sensitivity analysis at different thresholds did not improve the discrimination between the FiO2 parameter scores and the primary outcome. We have been unable to report the proportion of patients on a modified SpO2 scale, however because we used the NEWS2 score which already reflected the modified scale and because any bias related to a reduced FiO2 administration will have equally affected both the NEWS2 and NEWS-FiO2, this ought not to have affected the results. A complete case analysis approach was used which is justifiable as the rate of missing data was low and similarly distributed with the occurrence of the primary outcome. The fact that the findings are consistent across both the COVID-19 and influenza cohorts suggests that there is a limited impact of pandemic specific factors such as organisational strain, but this cannot be excluded.

Further work is needed to prospectively validate this model in an unselected hospital population, across multiple sites and in different demographic groups. In particular, the impact of bias in pulse oximetry and hence oxygen administration for patients with darker skin tones should be evaluated.26

Conclusion

Incorporation of FiO2 into NEWS2 improves predictive validity for adverse events for hospitalised patients with viral respiratory illness, particularly by improving the PPV. This benefit was mediated within those patients receiving oxygen supplementation and held true in both the COVID-19 and influenza cohorts. These findings require prospective validation across a more generalised population before consideration of inclusion in future iterations of NEWS2.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Acknowledgments

We wish to acknowledge the support of Leah Evans BS, Kathleen Merkley DNP APRN ANP and Samantha Johnson MBA, of Health Catalyst, Salt Lake City, Utah, USA, for developing the HealthCatalyst Data Operating System (DOS) used for this study. We would also like to acknowledge all of the clinicians and support staff who cared for patients during the COVID-19 pandemic.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors JC: conceptualisation, methodology, writing—original draft. JG: conceptualisation, methodology, writing—review and editing. DG: conceptualisation, methodology, validation, data curation. ND: conceptualisation, methodology, writing—review and editing. GG: conceptualisation, methodology, supervision, formal analysis, data curation writing, data visualisation—review and editing, guarantor. DG and GG have verified the underlying data. All authors have read and finally approved the version being submitted. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/disclosure-of-interest/ and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.