Automated audit of hospital oxygen use devised during the COVID-19 pandemic

Introduction

The UK National Health Service (NHS) has a long history of clinical audits and there is a mandatory requirement for hospitals to partake in some national-level audits. The National Clinical Audit and Patient Outcomes Programme involves audits commissioned by NHS England and comprises 23 national audits related to some of the most commonly occurring conditions.1 In addition to this, clinicians may undertake self-generated internal audit projects of their work and many Colleges and Societies commission specialty-related audits on a UK-wide basis. For example, the British Thoracic Society (BTS) arranges audits related to Respiratory Medicine on a regular or intermittent basis. One of these audits is the National Emergency Oxygen Audit which requires hospitals to report on several aspects of oxygen treatment including how many patients are using oxygen at the time of the audit, the oxygen device, the oxygen saturation and whether the patient has a prescription for oxygen use. The last such audit was conducted in 2015 and published in 2016 and is expected to be repeated soon.2 As is the case with most clinical audits, it is necessary for a team of clinicians to review the management of eligible patients. These audits are usually undertaken manually and require clinicians in each of the participating hospitals to visit every ward and unit to identify patients receiving oxygen at the time of the audit and then check the case records and prescription charts for these patients. This process was very time-intensive, often incomplete and subject to human errors, especially in view of the very large volume of information involved which is transcribed from clinical records into audit documents for each patient, and then amalgamated and summarised manually into the reports that are submitted to the BTS from each hospital.

In recent years, most NHS hospitals have changed from using paper medical records to electronic medical records (EMRs) in line with national initiatives such as the Wachter report.3 It has been suggested that some clinical audit processes can now be automated which will reduce the burden for clinicians and audit teams although the expertise of these teams must also be used and preserved.4 It has also been proposed that clinical audit can be combined with medical informatics to deliver ‘meta-audit’.5 In this hospital, all the clinical information that is required for some audits such as the BTS Emergency Oxygen Audit is captured electronically. This has offered us the opportunity to develop automated audits of oxygen use and other key aspects of patient care.

Methods

Salford Royal Hospital (now part of the Northern Care Alliance NHS Foundation Trust) is a 900-bed urban university hospital. The hospital has used the Sunrise EMR system since August 2000. The current version is Sunrise-Altera (formerly Sunrise-Allscripts) Release 18.4. This EMR routinely records demographic data and clinical data in addition to patient location within the hospital. Hospital policy since 2010 is that all in-patients should have a prescription with a target oxygen saturation range so that, in the event of deterioration, nursing staff will know what oxygen levels are appropriate for each patient.

Prescribing at Salford Royal is undertaken within the EMR. Oxygen prescriptions specify a target range of 92–96% for the majority of patients and a target range of 88–92% or an individualised target range for patients who are at risk of developing hypercapnia in response to oxygen therapy (most commonly patients with chronic obstructive pulmonary disease). The ‘standard’ target range for patients with no risk of hypercapnia was previously 94–98% in line with the BTS Oxygen Guideline prior to the COVID-19 pandemic in 2020.6 Since then, it has been changed to 92–96% in line with NHS Pandemic guidance and in line with the 2015 Thoracic Society of Australia and New Zealand guidance and the 2018 guideline by Siemieniuk et al which predated the pandemic.7–9 Manual audits from the introduction of online oxygen prescribing at Salford Royal in 2011 up to the BTS audit in 2015 have shown that about 90% of Salford patients using supplemental oxygen had an appropriate prescription, compared with an average of about 55% across the UK.2

Bedside observations for Salford Royal patients have been documented in the EMR since 2013, initially using the National Early Warning Score system (currently NEWS2) which was devised by the Royal College of Physicians and is used in most NHS hospitals.10 This system is used throughout the hospital in 56 wards, units and departments including the medical and surgical high dependency units but it is not used in the 36-bed critical care unit (CCU) which was therefore excluded from this audit.

Routine clinical observations are entered manually by bedside staff to capture the patient’s oxygen target saturation range, oxygen saturation measured by pulse oximetry (SpO₂), whether the patient is breathing air or oxygen (including details of oxygen devices and flow rates). Other routine observations include the patient’s heart rate, respiratory rate, blood pressure, temperature and consciousness level. These parameters are used to calculate an automated NEWS2 score within the EMR in accordance with the National Early Warning Score.10 Each of the above parameters scores 0 points if normal and up to 3 points if abnormal with a further 2 points added if the patient is using supplemental oxygen. Patients with NEWS2 scores of 5 and above require clinical review and patients with scores of 7 and above require review by a senior clinician.

The objective of this project was to automate the oxygen audit process which was previously undertaken manually. Following consultation between the respiratory department with the local Business Intelligence and IT teams in late 2019, a medical logic module was developed by the local IT team in early 2020. This produces an Excel spreadsheet, a summary table and a graphical display of the relevant data. The clinical information that is collected for each patient is shown in box 1.

A decision was made to report the final set of observations (closest to midnight) for the previous day. This avoids the risk of collecting data early in the day for large numbers of newly admitted patients who may not yet have any medicines prescribed or routine observations undertaken.

The automated oxygen audit project was at an advanced stage of development when the COVID-19 pandemic began in early 2020. Development of the project was accelerated because of the unique oxygen requirements caused by the pandemic and the system was first implemented in late March 2020. Initially, the business information team ran each audit when requested to do so by clinicians. The system was subsequently enhanced to allow authorised clinicians to create a new audit for the previous evening simply by opening the most recent audit in an Excel file and refreshing the data. The system was further enhanced in early 2023 to allow the business information team to sample the final observations set for every patient who had observations made during every day over the course of a given year. This allowed clinicians to view trends in oxygen prescribing as shown in the Results section.

Although the EMR records the oxygen flow for every patient using oxygen at the time of an observations set, we were unable to reliably estimate the total oxygen flow on each day because of the increasing use of HFNO (high flow nasal oxygen) and CPAP (continuous positive airway pressure) devices. These devices feed directly from the hospital’s piped oxygen and air supplies. The total flow from these devices can be above 100 L/min and oxygen flow can be 100 L/min or more but the screen on the devices does not display oxygen flows and air flows separately.

This was an observational audit study of routine clinical practice, not a clinical trial. The Trust Caldicott Guardian gave permission for data processing and for publication of the anonymous results. No statistical analysis was undertaken. There was no patient or public involvement in the study and there were no study participants, we used only anonymised numerical data related to routine bedside observations.

Results

The development and testing of the electronic audit tool extended from late 2019 to early 2020 and was finalised by March 2020 as described above. The audit was designed to capture all the clinical information described above for every current hospital inpatient within an Excel table. This allows clinicians to sort the data according to relevant parameters such as oxygen prescription type, oxygen therapy and oxygen saturation. Developing the database was quite a straightforward process for the technical team and only a few modifications and clarifications were required from the clinical team.

The system was designed to display an automated ‘dashboard’ on the first page of the Excel file. This shows the oxygen prescription status, inhaled gas (air or oxygen) and SpO₂ of every patient in the hospital as shown in figure 1. The dashboard is colour-coded to display information as follows:

• Green for patients with SpO₂ within their target range.

• Amber if the saturation is below the target range by 1–2% or above the target range by 1–4% while using supplemental oxygen.

• Red if the SpO₂ is below 86% or if it is above 96% for patients with a target range of 88–92% who are using supplemental oxygen and thus at risk from hypercapnia at saturations >92%.

• Below the colour-coded chart, there is an automated graphical display showing the use of oxygen according to the patient’s target saturation ranges (figure 2).

• Having identified patients of concern in the summary sheet, the audit team can open the ‘Raw Data’ sheet of the Excel file and drill down on individual patients, for example, those with hypoxia or hyperoxia. Using the hospital number, the audit team can then view the full clinical details in the EMRs for patients with unsatisfactory oxygen levels.

Request permission

Request permission

Periodic audits before, during and after the COVID-19 pandemic allowed clinicians to view the proportion of patients using oxygen on any given date and the number of patients with a prescribed target saturation range (figure 1). In the event of oxygen shortages due to surges in oxygen use, the system was designed to identify individual patients with potentially high oxygen flows. Should the total oxygen flow across the hospital come close to exceeding the capacity of the system (which did not happen), it would have been possible to identify areas in the hospital and individual patients with high flow rates and to initiate the appropriate response to increased demand. However, the individual oxygen flows for patients using HFNO and CPAP can be exaggerated as discussed in the Methods section so we have not included oxygen flow data in this publication.

Over 51 audits involving 34 352 patients, 2059 patients (6.0%) were using supplemental oxygen at the time of the observations set that was audited and 88.6% of those who were using oxygen had a valid prescription. The number of patients in hospital fell from about 800 to about 350 for a brief period during the first wave of the COVID-19 pandemic in spring 2020 and there was a smaller dip to about 550 during the second wave in Autumn 2020. The percentage of patients using oxygen increased during each wave of the COVID-19 pandemic. During the first wave of the COVID-19 pandemic in spring 2020, the monthly percentage of observations involving oxygen use increased from the baseline of 6.0% to a peak of 10.4% followed by a rise to 10.6% during the second wave and 7.4% during the third (Omicron) wave. (figure 3). Oxygen use returned to baseline after each wave.

Request permission

The system allowed the clinical lead for oxygen use (ROD) to identify individual patients who were receiving excessive amounts of oxygen and patients who were receiving oxygen with no prescribed target range. A clinical note was made in the EMR for these patients to indicate that their oxygen therapy and/or prescription status should be reviewed in line with the Trust oxygen policy.

Over 22 cycles of this intervention, 60 of 913 (6.6%) patients who were using oxygen had ‘iatrogenic hyperoxaemia’ at the time of the audit with either a target range of 92–96% but excessively high SpO₂ of 99–100% on supplemental oxygen or a target range of 88–92% with dangerously high SpO₂ of 96–100% on supplemental oxygen. This excessive oxygen therapy was either a one-off event or otherwise resolved within 24 hours in 55 of the 60 cases and five advisory clinical notes were made. Three of the five patients who had this prompt had satisfactory oxygen saturation on the following day and two were still receiving excessive oxygen. Thus 58 of 60 instances of excessive oxygen therapy were resolved within 48 hours.

The sporadic audits identified that 13.3% (121 of 913) of patients who were using supplemental oxygen did not have an oxygen prescription. Eighty of these 121 breaches of prescribing policy (66%) were resolved by the clinical team within 24 hours, mostly due to patients being discharged from hospital or ceasing to have oxygen therapy or having a prescription issued. For the remaining 41 patients using oxygen without a prescription, a clinical note was made to alert the clinical team to the use of oxygen without a prescription. 38 of these 41 instances were resolved within 24 hours of this prompt, mostly due to having a prescription (30 patients) or breathing air (6 patients). Only 3 of the 121 patients continued on oxygen without a prescription at 48 hours from the audited observations set. These data have been presented and discussed at the Trust Clinical Governance Board and the wards (mostly surgical) with the highest prevalence of unprescribed oxygen use have been identified.

Hypoxaemia (SpO₂ below 90%) was much less common than hyperoxaemia in these audits and did not require similar alerts because standard practice for nurses and doctors is to avoid hypoxaemia and the Salford EMR system already issues automated alerts for patients scoring 3 NEWS2 points due to low oxygen levels. Only 0.2% of 238 815 observations for patients with a target range of 92–96% in 2022 had hypoxaemia with SpO₂ <90% and only 0.05% had clinically concerning hypoxaemia with SpO₂ <85%. Spot audits of individual patients with severe hypoxaemia all showed that appropriate actions had been undertaken such as clinical review, resuscitation, respiratory review or a move to the CCU.

Discussion

In a hospital with a fully integrated EMR, we have shown that it is possible to automate all fundamental aspects of the BTS hospital oxygen audits. This allows senior clinicians to obtain frequent overviews of oxygen use both at individual patient level and across the hospital. This proved to be a valuable safety feature during the major waves of the COVID-19 pandemic because oxygen use from the non-critical care oxygen delivery system could be monitored easily, as well as remotely monitoring oxygen use at ward level and individual patient level.

Clinicians can also access raw data including ward location, oxygen device and flow rate for each individual patient and can thus alert clinical teams to patients who are receiving supplemental oxygen without a prescription or if there appears to be overuse of oxygen with saturation above the prescribed range, especially for patients who are at risk of developing hypercapnia if given too much oxygen. Hypoxaemia was much less common than hyperoxaemia in these audits and did not require intervention by the auditing team.

The most recent BTS UK-wide audit of oxygen use in hospitals was undertaken in 2015. 55 208 patients were audited in 180 hospitals and 14.0% were using oxygen which was much higher than the present Salford level of oxygen use (6.0%). An audit at a large American tertiary hospital in 2021 found that 19% of patients were using supplemental oxygen which is more than three times the level of use in Salford.11 The proportion of Salford patients using oxygen at the time of BTS oxygen audits was 18% in 2010, 11% in 2011, 10% in 2012 and 9% in 2015 with a further fall to 6% in the present audit. Salford has always been a low outlier in BTS audits because of strong local oxygen prescribing policies but there may have been similar downward drift in other UK hospitals since 2015 in the light of the COVID-19 pandemic when lower oxygen thresholds were set for UK hospitals (92–96% instead of 94–98%) and in light of the Chu and Siemieniuk papers in 2018 which cautioned strongly regarding the dangers of hyperoxaemia with a recommendation not to increase the SpO₂ above 96% with supplemental oxygen.9 12 13

The low level of oxygen use in Salford (compared with other UK hospitals) did not lead to a high incidence of hypoxaemia. Only 0.2% of observations for patients with a target range of 92–96% in 2022 had hypoxaemia (SpO₂ <90%) and only 0.05% had clinically concerning hypoxaemia with SpO₂ <85%. On the other hand, the more conservative use of oxygen in Salford was associated with a relatively low incidence of hyperoxaemia (6.6% of those using oxygen) and most of these episodes of hyperoxaemia were resolved quickly. Apart from the patient safety benefits of avoiding hyperoxaemia, there could be cost savings in terms of lower oxygen purchase costs for the hospital and staff time costs saved by the avoidance of having to deal with the complications of hyperoxaemia.9 13 There would be additional cost savings in undertaking automated oxygen audits. The automated audit takes about 2 min whereas we estimate that it took three specialist nurses up to 3 days to complete manual audits by visiting about 900 bedsides scattered across more than 50 wards, units and departments. The methodology described in this paper could be applied to any aspect of patient care where the relevant clinical data are gathered on a regular basis within an EMR. This model could be used to design automated audits for other common medical problems such as blood sugar levels in patients with diabetes, estimated glomerular filtration rate fluctuations in renal patients, monitoring of delirium, and potential applications for patients with high or low blood pressure or heart rates.

The strengths of this system are that it deploys a simple methodology which was quick to design and implement. Furthermore, it is possible to make rapid modifications to data collection or data display, for example, by the addition of NEWS2 scores alongside saturation data. The output is easy to understand and can be fed into serial audits to monitor trends in oxygen use over time. Individual audits can be run by selected clinicians in a matter of minutes and the lead clinician can decide which clinicians may have access to the data.

The main limitation of this system is that it can be implemented only in hospitals with fully integrated EMR prescribing and bedside observations systems; however, EMRs are being implemented rapidly in UK hospitals. The system is not designed to provide automated feedback to clinicians. Feedback is provided only if the auditing clinician makes a judgement based on serial observations because oxygen levels of individual patients may fluctuate rapidly. However, it would be possible to generate automated alerts if clinicians wished to do so. A further weakness is that audits are dependent on the enthusiasm and time capacity of individual clinicians.

In summary, we have described a relatively simple automated audit of oxygen prescribing and oxygen use that meets all of the requirements of the BTS oxygen audit. This system was also used opportunistically and successfully to monitor oxygen use during the COVID-19 pandemic. This method could be adopted to deliver automated audits for other treatments and patient monitoring by other specialties.