Article Text

Download PDFPDF

Variability of outcome measures in trials of intravenous therapy in acute severe paediatric asthma: a systematic review
  1. Charmaine S Gray1,
  2. Colin V E Powell2,3,4,
  3. Franz E Babl5,6,7,
  4. Stuart R Dalziel8,
  5. Simon Craig9,10
  6. on behalf of the PREDICT (Paediatric Research In Emergency Departments International Collaborative) Research Network
  1. 1 Emergency Department, Women’s and Children’s Hospital, University of Adelaide, North Adelaide, South Australia, Australia
  2. 2 Emergency Department, Sidra Medical and Research Center, Doha, Qatar
  3. 3 Emergency Department, Weill Cornell University, Doha, Qatar
  4. 4 School of Medicine, Cardiff University, Cardiff, UK
  5. 5 Emergency Department, Royal Children’s Hospital, Melbourne, Victoria, Australia
  6. 6 Murdoch Children’s Research Institute, Melbourne, Victoria, Australia
  7. 7 University of Melbourne, Melbourne, Victoria, Australia
  8. 8 Children’s Emergency Department, Starship Children’s Hospital, Auckland, New Zealand
  9. 9 Monash Emergency Research Collaborative, School of Clinical Sciences at Monash Health, Monash University, Clayton, Victoria, Australia
  10. 10 Murdoch Children’s Research Institute, Melbourne, Victoria, Australia
  1. Correspondence to Dr Charmaine S Gray, Emergency Department, Women and Children’s Hospital, North Adelaide, SA 5006, Australia; charmaine.gray2{at}sa.gov.au

Abstract

Objective To determine the variability of primary and secondary outcomes used in trials of intravenous bronchodilators in children with acute severe paediatric asthma.

Methods Systematic search of MEDLINE, EMBASE, Cochrane CENTRAL and the WHO International Clinical Trials Registry Platform for randomised trials in children (less than18 years) with acute severe paediatric asthma comparing intravenous bronchodilator therapy to another treatment. Initial search was performed on 7 January 2016 with an updated search performed on 6 September 2018. Primary and secondary outcomes were collated.

Results We identified 35 published papers and four registered study protocols. 56 primary outcomes were found, the most common being a clinical asthma score (23/56; 41%). Other identified primary outcomes included bedside tests of respiratory function (11/56; 20%) and measures of length of stay (9/56; 16%). There were a total of 60 different secondary outcomes, the most common were various length of stay measures (24/60; 40%) and adverse events (11/60; 18%).

Conclusion Studies comparing intravenous treatment modalities for children with acute severe paediatric asthma exhibit great variation in the type, number and timing of outcome measures used. There are no patient or family-specific outcomes reported. There is a need to develop international consensus.

Trial registration number CRD42017055331.

  • paediatric emergency med
  • respiratory, asthma

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Key messages

What is already known on this subject

  • Despite the fact that there are a number of randomised trials on intravenous therapy for paediatric severe asthma, there remains considerable variation in treatment.

  • Variation in outcome measures and lack of patient-centred outcomes are likely to contribute to the uncertainties regarding this treatment.

What this study adds

  • In this systematic review of randomised studies of intravenous bronchodilator therapy we found 56 different measures of primary outcome, the most common being a clinical asthma score (23/56; 41%), with pulmonary function tests (11/56; 20%) and length of stay measures (9/56; 16%) also featuring highly.

  • Few studies considered health economic data (2/39; 5%) and none considered clinician and patient perspectives.

  • The development of a core outcome set is required to allow for meaningful, comparable studies to be conducted.

Introduction

There is considerable uncertainty regarding the best intravenous agent to use for treatment of acute severe paediatric asthma.1 2 This uncertainty is due to a lack of robust evidence, a large existing variation in practice,2 differences in the definition of severe acute asthma and differing thresholds to initiate treatment.

Current evidence of variation in treatment of acute severe asthma in children includes a recent UK and Ireland study, which identified more than 30 different intravenous treatment regimens in 110 children presenting to 24 paediatric EDs.3 Similarly, an Australasian survey of paediatric emergency clinicians demonstrated highly variable practice in self-reported treatment of severe to critical asthma in children.4

In addition to this large variation in practice, evidence from completed studies is difficult to compare due to differences in outcome measures between trials.2 5 6 No current consensus guidelines on the conduct and reporting of trials in acute severe paediatric asthma exist. Patient-reported outcomes and those related to health economics are limited. Previous Cochrane reviews have recommended consensus on core outcomes7 and a need to focus on clinically important outcomes such as admission to intensive care, hospital admission, length of stay and relapse rates.8

This systematic review aims to determine the variability of current primary and secondary outcomes used in comparative studies of intravenous bronchodilators for acute severe paediatric asthma. It is hoped this will lay the foundation for further discussion around the utility of these outcomes, and how the research community could move towards a useful, inclusive, defined set of core research outcomes for this group of patients.

Methods

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.9

Eligibility criteria

Studies were deemed eligible if they included children (defined as younger than 18 years) and had a randomised trial design comparing an intravenous bronchodilator treatment to another bronchodilator or to placebo in acute severe asthma. All languages were considered. For the search of the WHO clinical trials database, trials that were registered even if not yet active were included. As our research question focused on acute severe asthma in children, we excluded studies that included a combination of adults and children where the paediatric patients were unable to be separated from the adult patients.

Information sources

We systematically searched MEDLINE, EMBASE, Cochrane CENTRAL databases and the WHO International Clinical Trials Registry Platform for randomised trials in children (aged 1–18 years) with acute severe asthma. Search was performed on 7 January 2017 and repeated on 6 September 2018 to identify any new studies prior to publication of the review. All databases were searched from inception of database. A full search strategy for each database is listed in online supplementary table 1.

Supplemental material

Study selection

Search results were reviewed independently by two authors (CSG and SC) and those deemed not to meet inclusion criteria by title and abstract were excluded. Full-text articles were then reviewed to determine eligibility. Any disagreement was resolved by discussion and reaching consensus. For all relevant Cochrane reviews and meta-analyses, individual studies were obtained, and assessed against our inclusion criteria. Two non-English papers were translated using Google Translate (Google, Mountain View, California, USA), with information verified from clinical trial registries where possible.

Data collection process

The following characteristics were recorded for each study: geographic location, year of publication, number of patients, primary, secondary and other outcomes used, study interventions including dosing and timing of medication administered.

These data were collected independently by two authors and were recorded on a Microsoft Excel spreadsheet (Excel 2008; Microsoft; Redmond, Washington, USA). Any disagreement regarding extraction of data or outcome categorisation was resolved by discussion. For all studies that reported a clinical asthma score, the Methods section of the paper was reviewed to identify the specific asthma clinical score used, and the timing and cut-off used to determine patient improvement was recorded. In addition, all outcome measures were recorded as listed in the paper; if the paper reported two primary outcomes these were both recorded as primary outcomes and used in data analysis. Given that this systematic review was designed solely to describe and analyse outcome measures there were no data collected on the design or the quality of the study.

Results

From the 1293 studies identified across the databases, 197 duplicates were excluded; review of titles and abstracts yielded 89 papers that were selected for full-text review. Thirty-five published papers and four registered study protocols were included for analysis. The selection of articles is outlined in the PRISMA diagram (figure 1).

The 35 published studies ranged in date from 1971 to 2017, with ongoing recruitment of patients in the four registered study protocols. The number of patients within the completed studies ranged from 21 to 276 with a median (IQR) of 44 (29–56), and involved eight different medications (online supplementary table 2). Of the studies, 10 (29%) did not include any teenage patients, defined as being of 12–18 years, and 10 (29%) did not include any preschool patients, defined as being less than 6 years. Medication doses and administration varied between studies, with the same medication being given by bolus or infusion, at different time intervals, or for different infusion durations. Aminophylline, which is featured in 12 randomised trials, did not have any two trials which used the same dosage or timing of medication. Two aminophylline studies did not specify details of drug administration (online supplementary table 3). There was also a difference in the capturing of adverse events with 6/13 (45%) trials involving magnesium sulfate including this as an outcome measure, compared with 9/12 (75%) and 4/4 (100%) studies involving aminophylline and theophylline, respectively (online supplementary table 4).

There were a total of 56 primary outcomes listed across the 39 studies. The majority (23/38; 60%) of studies listed one primary outcome. Eleven studies (29%) reported two primary outcomes, four (11%) reported three primary outcomes and one study (3%) reported four primary outcomes. The most commonly used primary outcome was a clinical asthma score (23/56; 41%). Other identified primary outcomes included bedside tests of respiratory function (11/56; 35%), and various length of stay measures (9/56; 16%), including hospital, ED or intensive care length of stay. Two trials (2/56; 4%) included oxygen saturation. The majority of secondary and other outcomes listed were length of stay measures and adverse events. No study included a patient-reported variable as an outcome measure and only two studies incorporated cost-effectiveness measures. The frequency of primary and secondary outcomes is outlined in table 1. A full list of outcomes measures by study is presented in the online supplementary table 5.

Table 1

Frequency of primary and secondary outcome measures

There were at least eight different clinical asthma scores used (table 2); in addition, six studies described use of an asthma score but did not provide sufficient detail to determine which score was used. Timing of measurement varied. Scores that had the same label (eg, the ‘clinical asthma score modified from the pulmonary index’) did not necessarily measure the same components. In addition, within the same score, the cut-off used to define success of the administered treatment varied between studies.

Table 2

Clinical asthma scores used as primary outcome measures

Secondary outcomes for most studies comprised more than one measure with a total of 60 outcomes identified across the 39 studies. The most common secondary outcome measures related to hospital length of stay (24/60; 40%) and adverse events (11/60; 18%). Additional outcomes included those relating to clinical asthma scores, bedside pulmonary function tests, frequency or type of asthma medications administered, physical examination findings and vital signs.

Discussion

This systematic review demonstrates that studies of intravenous bronchodilator therapy for children with acute severe paediatric asthma have used inconsistent and variable outcome measures. Therefore, uncertainty exists for clinicians who work in an ED about which intravenous treatment to use and the optimum regime. Additionally, we do not know what is most important to patients and families, and have little information to explain how the treatment we are providing will benefit them. Finally, we are unable, with current data, to establish or compare the cost-effectiveness of varying treatment regimes. In an uncommon condition, such as acute severe paediatric asthma, creating reliable data is dependent on the ability for study results to be compared and combined in systematic reviews and meta-analyses; currently, the lack of comparable outcomes makes this very difficult.

The most commonly recorded primary outcome was a clinical asthma score. A valid and reliable bedside clinical assessment of severity in children is potentially useful, as it allows for quantification of severity of illness and is a way to measure improvement. In addition, most international asthma guidelines suggest use of clinical scores to assess severity and guide treatment response, although it is noteworthy that none recommends a specific score.10 11A recent systematic review of clinical scores for the assessment of acute dyspnoea in wheezing children found that none of over 20 scores examined had been sufficiently validated to allow for clinically meaningful use in this population.12 The Cochrane review by Mitra and colleagues reiterates these concerns, commenting that the changes in clinical scores remain largely invalidated against changes in lung function.8 As such, further studies are required to improve the evidence base supporting bedside scores of childhood asthma, with a view to reducing the number of different scores currently used, and achieving better understanding of the clinical relevance of such scores.12

We also suggest that health economic measures be incorporated into the core outcome set (COS). Only two studies considered health economics data as an outcome.13 14 Wheeler and colleagues compared medication and blood-level monitoring costs,13 and Irazuzta and colleagues calculated savings based on earlier discharge in the treatment arm.14 Assessing the cost of the treatment is likely to become increasingly important in any study design. This is highlighted in a recent paper by Petrou and colleagues15 which estimated the cost-effectiveness of nebulised magnesium sulfate in addition to usual care in acute paediatric asthma.16 A method to incorporate economic value outcomes alongside clinical ones has been proposed by Ramsey and colleagues.17

Other important measures of the effect of acute asthma treatment include hospital length of stay and intensive care unit admission. However, in the current studies, various measures were employed, including actual time to discharge and time for readiness for discharge, both for ED, hospital and intensive care stays. The definition of the time for readiness for discharge also varied between studies. With such variability, some agreement on length of stay parameters that are important to both clinicians and patients is needed.

We therefore suggest that a core set of outcomes be developed. This drive for consistent outcomes in clinical trials is reflected in the wider global research community.18 The Core Outcome Measures in Effectiveness Trials initiative is a programme that promotes the development of COS and a practical guideline on how to develop such core outcomes has recently been. published.19 20 There are some examples of illnesses where COS have already been developed in the paediatric population including rheumatoid arthritis21 and eczema.22

In addition, obtaining the views of patients, caregivers and clinicians is vital, and there have been recent publications focusing on both the importance of, and how to include, patient-focused outcomes. This is exemplified by the recent extension of the Consolidated Standards of Reporting Trials (CONSORT) statement, the CONSORT Patient Reported Outcomes.23 To our knowledge, there is no literature on including patient-focused outcomes in acute severe paediatric asthma and no studies in our systematic review included such outcomes.

Some literature exists in the area of care of children who use regular asthma medications as reported by Sinha and colleagues.24 They used a modified Delphi process to identify outcomes relevant to clinicians, parents and young people, and found that daytime and nocturnal symptoms, exacerbations, quality of life and mortality were important outcomes. Although these outcomes focus on treatment success with preventer medication, they may have some applicability to children presenting with acute severe paediatric asthma. However, other outcomes may be more pertinent in the acute setting, including medication side effects, need for interhospital transfer, intensive care admission and/or mechanical ventilation, healthcare costs and duration of hospitalisation. To progress research in this area, we suggest that the views of patients, carers and families be obtained using interviews, focus groups and/or surveys,25 in order to develop a set of core outcomes in acute severe paediatric asthma.

Limitations

This systematic review is limited to studies of children with acute severe paediatric asthma requiring intravenous bronchodilator therapy. It was beyond the scope of our review to determine comparative effectiveness of asthma therapies. Recently, a protocol for an overview of Cochrane reviews for interventions for escalation of therapy for acute exacerbations of asthma in children was published.26 This may provide more information regarding comparative effectiveness of various acute asthma treatments in this setting.

There is potential selection bias relating to study inclusion, which was minimised by two authors selecting studies independently and resolving differences by consensus. Data collection was also completed independently, by clearly defined methods, which were registered prior to data analysis. This review might also be affected by publication bias with only positive studies being published in trials on comparative intravenous therapy, although we attempted to reduce the impact of such bias by conducting additional searches in relevant clinical trial registries.

Conclusion

Trials of intravenous bronchodilator therapy for acute severe paediatric asthma demonstrate significant variability in major outcome measures including choice of asthma clinical scores, length of stay parameters, limited consideration of health economics and an absence of patient-focused outcomes. Given this heterogeneity we recommend development of a COS that can be used in trials of intravenous therapy for acute severe paediatric asthma. This will inform the design of subsequent randomised trials, which would allow for meaningful comparisons between therapies for acute severe paediatric asthma.

Acknowledgments

The PREDICT network acknowledges the assistance in accessing databases and article acquisition of Ms Natalie Dempster, Librarian at the Women’s and Children’s Health Network Library, Women’s and Children’s Hospital Adelaide.

References

Footnotes

  • Contributors The literature search, data collection, analysis and writing of the paper were performed by CSG and SC. Comments and editing were performed by coauthors FEB, SRD and CVEP.

  • Funding FEB has been supported by a practitioner fellowship grant from the National Health and Medical Research Foundation, Canberra, Australia, and the Melbourne Children’s Campus Clinician Scientist Fellowship, Parkville, Australia. SRD’s time was part funded by the Health Research Council of New Zealand (HRC13/556). This research was supported by a National Health and Medical Research Council Centre of Research Excellence grant for paediatric emergency medicine (GNT1058560).

  • Competing interests None declared.

  • Patient consent Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement There are no additional unpublished data.