Respiratory Research

Objective measurement of lung volume recruitment therapy: laboratory and clinical validation

Abstract

Lung volume recruitment manoeuvres are often prescribed to maintain respiratory health in neuromuscular disease. Unfortunately, no current system accurately records delivered dose. This study determined the performance characteristics of a novel, objective, manual lung volume recruitment bag counter (‘the counter’) with bench and healthy volunteer testing, as well as in individuals with neuromuscular disease. We undertook (1) bench test determination of activation threshold, (2) bench and healthy volunteer fidelity testing during simulated patient interface leak and different pressure compressions and (3) comparisons with self-report in individuals with neuromuscular disease. The data are reported as summary statistics, compression counts, percentage of recorded versus delivered compressions and concordance (Cohen’s kappa (K) and absolute agreement).

Results Minimum counter activation pressure under conditions of zero leak was 1.9±0.4 cm H2O. No difference was observed between the number of repetitions delivered and recorded during high airway pressure condition. Interface leak approximating 25% resulted in underestimation of repetition counts, and once the leak was at 50% or beyond, the counter recorded no activity. Faster sampling frequency collected data with more fidelity. Counter data agreed with diary self-report during community trials (16 participants, 960 participant days, 77% agreement, Cohen’s Κ=0.66 and p<0.001). Disagreement typically favoured more diary reported (18%) than counter (5%) sessions.

Conclusions The performance characteristics of a new lung volume recruitment counter have been established in both laboratory and community settings. Objective usage and dosage data should accelerate new knowledge development and better translation of lung volume recruitment therapy into policy and practice.

Key messages

What is the key question?

  • What are the key performance characteristics of a novel adherence meter for lung volume recruitment?

What is the bottom line?

  • The device provides a pathway for accelerated clinical uptake of lung volume recruitment by performing a series of bench tests in normal controls and during a randomised controlled trial.

Why read on?

  • This novel device provides health professionals for the first time with an effective method of measuring patient adherence and the delivered dose of lung volume therapy.

Introduction

Many neuromuscular diseases (NMDs) result in progressive weakness of the respiratory muscles, which leads to decreases in lung capacity and a reduced ability to cough effectively. Respiratory failure secondary to respiratory muscle weakness, chest wall restriction, impaired cough and recurrent respiratory tract infections is the predominant cause of morbidity and mortality in this population.1 Lung volume recruitment (LVR) is a technique frequently used by people with NMD to achieve larger lung volumes and enhance cough. Consecutive ‘breaths’ of inspiratory pressure are delivered to the airways via a manual self-inflating bag (‘resuscitation bag’), with or without a one-way valve, attached to an oronasal mask or mouthpiece. These consecutive, assisted breaths stack to a lung inflation volume greater than that achievable with spontaneous breathing. Previous research has demonstrated that LVR increases maximum insufflation capacities (MIC) or lung insufflation capacity (LIC)1–9 and peak cough flow during the LVR.4 7–17 Retrospective and cohort studies in NMD suggest that LVR may slow the decline in lung function over time.2 3 18

Despite a large literature base that recommends routine LVR for people with NMD,2 3 5 6 9 19–25 little is known about how people actually use LVR therapy nor what is the optimal, effective dose. Some studies have tracked self-reported adherence to prescribed LVR therapy using diaries6 20 or patient recall.21 However, self-reporting has been shown to overestimate adherence to airway clearance techniques26 and prescribed medication27 in other chronic disease populations. Prescribed LVR dosage in the literature varies from two to four times per day2 3 5 6 9 19–22 with inconsistent relationships reported between self-reported ‘dose’ and outcomes. To determine how often people use LVR and thus establish dose–response relationships, a robust, objective measure of usage is necessary.

The primary aim of this study was thus to describe the design and assess the performance characteristics of a custom-made device (‘LVR counter’) that objectively records LVR usage. To address this aim, a series of laboratory-based tests were undertaken to determine the technical capacity of the LVR counter to:

  • Determine the minimum pressure threshold that triggered LVR counter activation.

  • Record LVR compressions and repetitions under low and high airway pressure conditions.

  • Record LVR usage when performed with an adequate seal at the patient interface, defined as ≤50% leak.

Performance of the LVR counter when recording from human participants examined:

  • The optimal data sampling frequency using a physiological control subject (healthy volunteer).

  • The performance of the LVR counter ‘in the field’ by comparing LVR counter downloads with participant self-report during a randomised controlled clinical trial of individuals with neuromuscular disease.

Methods

Equipment: LVR counter

The LVR counter, engineered by the rehabilitation engineering department at the Ottawa Hospital Rehabilitation Centre (Ontario, Canada), was designed to be attached to a commercially available LVR kit (1.6 L self-inflating bag, one-way valve, tubing and mouthpiece or oronasal mask, item number 1034502; Mercury Medical, Florida, USA; figure 1). The LVR counter comprised a battery-powered state data logger (Omega OM-CP-State101A data logger, OMEGA Engineering, Stamford, Connecticut, USA) wired to two pressure switches connected in series (Model 7411–711, PSF102 Series pressure switch, DesignFlex Switches, World Magnetics Company, Traverse City, Michigan, USA). One pressure switch was connected to the LVR bag while the other connected near the patient interface via 3 mm silicone tubing (figure 1 and online supplemental file 1). The state data logger registered a count only when both pressure switches reached minimum thresholds. This design element minimised false-positive recordings, which may occur, for example, with bag compressions in the presence of excessive patient interface leak or with no person at the interface. As such, the LVR counter was designed to be analogous to ‘time at pressure’ usage recording as used in continuous positive airway pressure therapy and thus to only record compression when both the bag was compressed and the LVR circuit interfaced with a patient. A cable (OM-CP-IFC200 USB and stereo jack connector, OMEGA Engineering) connected the state data logger to a computer running proprietary Omega software (OM-CP Data Logger Software, V.4.2.7.0, OMEGA Engineering). The software set both the data logger recording parameters (sampling rate, auto on/of and so on) and allowed for the export of counter data that had been recorded over time.

Figure 1
Figure 1

Illustration of the lung volume recruitment (LVR) counter (insert), attached to a commercially available LVR kit. The LVR counter is composed of a battery-powered state data logger (Omega OM-CP-State101A data logger, OMEGA Engineering, Stamford, Connecticut, USA) wired to two pressure switches connected in series (Model 7411–711, PSF102 Series pressure switch, DesignFlex Switches, World Magnetics Company, Traverse City, Michigan, USA). The LVR kit pictured includes a 1.6 L self-inflating bag, one-way valve, tubing and mouthpiece (item number 1034502; Mercury Medical, Florida, USA). Additional 3 mm silicone tubing connects one pressure switch to the LVR kit near the patient interface (via existing port on on-way valve). The second pressure switch connects directly to the LVR bag via an existing port (insert).

To ensure that the ‘dose’ in the following experiments was reproducible, standardised definitions were used (table 1 and figure 2). In particular, the term LIC rather than MIC, as defined in a recent consensus document, was employed because the circuit includes a one-way valve, and as such, participant glottic control was not required.18 28 Much like a weight-training regime, the LVR ‘dose’ is composed of number of ‘sessions’ per day, where each ‘session’ is a number of ‘sets’ of ‘repetitions’. Each ‘repetition’ is one complete assisted inflation to LIC or MIC and is composed of consecutive ‘compressions’ of the LVR bag. The number of ‘compressions’ to achieve a ‘repetition’ depends on patient comfort, respiratory system compliance, glottic function and so on.

Table 1
|
Definitions used in lung volume recruitment (LVR) therapy
Figure 2
Figure 2

Schematic representation of lung volume recruitment (LVR) dose. The LVR ‘dose’ is composed of a number of ‘sessions’ per day, with each ‘session’ consisting of ‘sets’ of ‘repetitions’. A ‘repetition’ is defined as one complete assisted inflation to maximal, tolerable, insufflation capacity. The number of consecutive ‘compressions’ or ‘insufflations’ of the LVR self-inflating bag to achieve one assisted inflation varies for each individual. This example depicts an LVR dose of three sets of three repetitions, performed twice a day. Set 1 of session 1 also illustrates the protocol for Experiment 4, whereby the number of compressions performed per repetition increased over the set (first repetition=two compressions, second repetition=three compressions and third repetition=four compressions). This was repeated for each sampling rate tested (see text for more information).

Patient and public involvement

Patients or the public were not involved in the design, conduct, reporting or dissemination plans of this research project; public and patient review of research is provided through membership of the local human research ethics committee.

Experiment 1: minimum compression pressure required to trigger LVR counter (and generate a counted event)

The LVR circuit was connected directly to a Micro Respiratory Pressure Meter (MicroRPM, CareFusion, San Diego, California, USA) at the patient end, mimicking ideal clinical conditions (ie, bag compression and complete seal at the mouthpiece). Data sampling frequency was set at 1 hertz (Hz) for all testing. For each trial, the LVR bag was compressed slowly until the LVR counter recorded an event. The pressure threshold required to activate the LVR counter was recorded, and 50 compressions were performed overall.

Experiment 2: performance of the LVR counter during simulated near-complete lung inflation

Airway pressure increases substantially during LVR, and to determine if the LVR counter continued to operate under higher pressure conditions, the mouthpiece was removed and the LVR circuit attached at the patient end to a 1 L test lung (Maquet Adult 1 Litre Test Lung, item number 6006832, Getinge, New Jersey, USA). Four sets of 10 repetitions (total n=40) were administered; each repetition comprised three compressions (120 compressions in total). Between each repetition, the test circuit was disconnected to return the circuit to atmospheric pressure. Pressures up to 100 cm H20 were delivered to mimic high-resistance (low respiratory system compliance) conditions.

Experiment 3: assessment of the effect of simulated mouth leak on counter performance

To test whether the LVR counter only recorded a compression when there was an adequate seal at the mouthpiece, 100 compressions were performed in each of the following simulated leak conditions, achieved by covering the mouthpiece outlet to varying degrees:

  1. Zero leak: complete mouthpiece occlusion.

  2. Minimal (~25%) leak: three-quarters of the mouthpiece covered.

  3. Moderate (~50%) leak: half of the mouthpiece covered.

  4. Large (~75%) leak: one-quarter of the mouthpiece covered.

Experiment 4: LVR counter accuracy at different sampling frequencies and with different bag compression speeds

The accuracy of the LVR counter to record compressions and of the software to thus determine repetitions of LVR therapy on a healthy volunteer was determined at three data logger sampling rates (4, 2 and 1 Hz). At each sampling frequency, three sets of LVR with three repetitions but variable number of compressions were performed: two, three and four compressions per repetition (figure 2). The speed of compression increased across the sets.

An unblinded research assistant programmed the sampling rate into the LVR counter. The order for presentation of the sampling frequencies was randomly determined (www.randomization.com). The clinician performing LVR compressions was thus blinded to the sampling frequency of each test. The number of compressions and repetitions was compared with the actual delivered (27 compressions each composed of three sets of three repetitions, figure 2).

Experiment 5: LVR usage in a community setting

The performance of the LVR counter in a community setting was examined using data from a clinical research trial. Concordance between two methods of assessing therapy usage was assessed in participants in the intervention arm of a randomised controlled trial (‘Lung Volume Recruitment in Neuromuscular Disease: Can “breath-stacking” improve lung function, respiratory symptoms and quality of life for people with neuromuscular disease?’ Australian New Zealand Clinical Trials Registry ACTRN12615000565549).

Participants were prescribed a LVR therapy dose of ‘at least two sessions daily’. These LVR treatments were administered by the patient, her/his family or carer as clinically appropriate. Each LVR session comprised up to five sets of five LVR repetitions separated by quiet breathing. Study participants used both self-report diaries and the LVR counter to record the number of LVR sessions completed per day. The LVR counter was set to sample at 1 Hz, and the time and date-stamped compression data were summarised into number and duration of sessions per day using an Excel macro (table 1 and figure 2; Excel example and associated data download instruction at https://doi.org/10.5281/zenodo.4587974).29

The LVR counter records compressions to memory. These compressions are combined to calculate repetitions, sets, sessions and dose as illustrated in figure 2.

Daily session data were paired by date, for days 15–75 of the 90-day study period. This mid 60-day period was selected to better represent ‘normal’ daily use, as it was hypothesised that participants’ behaviour may be influenced immediately preclinical or postclinical trial outcome assessments. The difference between the reported (diary) and recorded (LVR counter) sessions per day and per participant was calculated. Where LVR data matched the self-reported information, this was recorded as ‘0’. Where there was a discrepancy in the paired data, this was categorised as ‘positive diary’ or ‘positive LVR counter’. Concordance was analysed for 16 consecutive participants. No formal sample size calculation was conducted.

Statistical analysis

Summary statistics for activation pressures are presented as mean±SD values. The number of compressions delivered was counted manually and compared with that recorded by the LVR counter. Percent error was calculated as (LVR count/manual count * 100) and a 95% CI obtained using the Wilson score method. Accuracy of the LVR counter to record at different sampling rates was expressed as the percentage of actual delivered. Concordance between self-report and LVR-recorded use was reported as both a weighted Cohen’s kappa (Κ) and as absolute agreement.

Results

Experiments 1 and 2: performance at low and high pressures

Forty-nine of the delivered 50 compressions were recorded by the LVR counter at low pressure, resulting in a 2% error rate (95% CI=0.4 to 11). The minimum pressure threshold required to activate the device under conditions of zero leak was 1.9±0.4 cm H2O. Under conditions of higher pressure, the LVR counter recorded more compressions than were delivered (mean (95% CI) error=+13% (8 to 21). Despite this overcounting of compressions, the number of repetitions was accurately recorded by the LVR counter as 40 (table 2).

Table 2
|
Manual (ie, delivered) and LVR counter recorded counts for compressions and repetitions, for each experiment

Experiment 3: performance in the presence of leak

The LVR counter overestimated the number of single compressions delivered when zero leak was simulated and underestimated in the presence of minimal leak. If significant leak was introduced (≥50%), the LVR counter failed to record compressions (table 2).

Experiment 4: LVR counter accuracy in a healthy control participant

The performance of the LVR counter sampling at 1, 2 and 4 Hz in a healthy volunteer is reported in table 3. Under all conditions, the device was able to detect that a session of LVR, comprising three sets of three repetitions, had been conducted; however, individual compressions were missed at lower sampling frequencies.

Table 3
|
Number of compressions recorded by the LVR counter, at three different sampling rate conditions, in a healthy volunteer

Experiment 5: LVR ‘in the field’

Data from 16 participants (960 possible participant days) were analysed. A total of 933 diary reports and 880 LVR counter records were collected, with paired data available for 853 participant days. One participant lost his LVR kit, and two LVR counters malfunctioned for a period of time. All participants or their carers were able to successfully turn the LVR counter on and off.

There was substantial agreement between the number of sessions recorded per day (77% agreement, weighted Cohen’s Κ=0.66, p<0.0005; table 4). The highest concordance was demonstrated when two sessions per day were recorded in both the diary and the LVR counter. Disagreement typically favoured more sessions reported in the diary (18%) versus LVR counter recordings (5%).

Table 4
|
Agreement matrix of lung volume recruitment (LVR) therapy sessions recorded by the LVR counter compared with participant diary, for 16 people with neuromuscular disease in a community setting

Discussion

This study describes the validation and use of an LVR counter, retrofitted onto a commercially available LVR kit, to objectively quantify an individual’s concordance with prescribed LVR therapy. The minimum pressure required to trigger the LVR counter’s pressure switches under conditions of zero leak was 1.9 cm H2O, suggesting that it is extremely sensitive in detecting usage (Experiment 1). The LVR counter is accurate at detecting single compressions of the LVR bag in a controlled setting, with the LVR counter recording 98% of all compressions at low pressure (Experiment 2). The in-series design of the LVR counter ensured that both a compression of the bag and adequate participant seal must simultaneously occur for use to be recorded (Experiment 3).

Although the LVR counter overestimated the count of individual compressions delivered to the test lung under high-pressure bench test conditions (up to 100 cm H2O) by 15%, it demonstrated 100% accuracy in counting all 40 repetitions. Arguably, the ‘overcounting’ of compressions is of little clinical importance because compressions are simply the mechanism by which one achieves the clinically important outcome; the attainment of the therapeutically targeted maximal tolerated insufflation capacity (MIC or LIC); and the repetition, sets and sessions that sum to provide the daily dose of LVR therapy (figure 2). Physiological control testing (Experiment 4) similarly demonstrated 100% accuracy between repetitions and sets of LVR therapy conducted and recorded, regardless of the speed at which the technique was performed. Increasing the LVR counter’s sampling rate to 4 Hz recorded compressions precisely, and therefore, if required perhaps for research applications, both compressions and repetitions may be recorded with higher fidelity.

The use of this LVR counter in the community appeared feasible; data were obtained for 92% participant days (880 out of a possible 960). Of the 853 sets of paired data, there was absolute agreement regarding the daily dose between the LVR counter and self-report diary 77% of the time (653 instances). Greater discordance in the direction of the diary suggests overestimation of self-reported LVR usage, a common finding in other chronic conditions when self-report measures have been compared with objective measures of use.26 27 In one study, 47% of people with cystic fibrosis reported performing more airway clearance therapy sessions than objectively recorded data would suggest across a 5-week period.26 In contrast, medication adherence literature would indicate better concordance, with 72% of studies reporting ‘high’ concordance; high concordance is defined as <10% difference between self-reported diary and other measures of adherence.27 Differences in concordance rates between self-reported and objective measures may be related to the type of intervention (eg, ease and time required to perform), duration of follow-up period and participant awareness of monitoring.

A limitation of the way the LVR counter was used in our field testing was that it needed it to be manually switched ‘on’ before each therapy session. This approach was taken to ensure adequate battery life as the LVR counters’ data were downloaded only once a month. As such, forgetting to start the counter before each therapy session would lead to an under-recording of LVR usage and could partially explain the apparent overestimation on the diary. Anecdotal participant reports confirmed this, but there was no suggestion this accounted for all discordant data. In 5% of cases, we identified a therapy session had been recorded on the LVR counter and not documented in the diary. Given the LVR counter’s design, we suggest that when discordance favours the LVR counter, this represents actual usage and a conservative but accurate representation of ‘at least’ delivered dose.

The ability to objectively monitor therapy use, dosage or concordance with recommended prescription is much needed; examining dose–response is vital to assess the efficacy of LVR therapy and establish empirically derived optimal dose recommendations. Moreover, monitoring of treatment data including therapy use has been shown to improve adherence to continuous positive airway pressure treatment, when part of a multimodal intervention strategy in people with obstructive sleep apnoea.30 To date, evidence of effectiveness of LVR is largely limited to retrospective cohort design or prospective studies with only participant self-report of use. In other conditions, this has been shown to be inaccurate, and hence, translation of these studies into practice and policy is compromised. Reports of little or no therapeutic benefit may reflect non-performance of treatment or a true lack of effect, but without objective usage data, it is not possible to know. The ability to correlate clinically important outcomes with a specific threshold of utilisation will assist in clinical prescription so as not to overburden or undertreat individuals. The requirement for both pressure switches to be triggered during the manoeuvre is a strength of the design of this LVR counter. This guards against usage being recorded when the LVR bag is compressed but not connected to the person (ie, simulating usage or demonstrating the equipment). Similarly, ineffective therapy due to interface leak is not counted as demonstrated by the lack of any recorded compressions in conditions with substantial leak.

Limitations

There was no recording of the flow or volume delivered during LVR nor any external criterion measure of use during field testing (Experiment 5); we were comparing the new LVR counter to self-report. While the performance of the LVR counter in the preceding bench tests strongly suggests that the LVR counter accurately records repetitions, sets and thus sessions and daily dose, there is a possibility that the self-report data were ‘more true’ than the LVR counter. As noted, previous literature in other self-report versus objective measurement conditions would make this possibility unlikely.26 27 The LVR counter used in this project is an in-house-developed prototype and as such does not provide a clear pathway to commercial and clinical translation of our findings; however, we suggest that this paper illustrates clear translational opportunities. Clinically, objective data provide opportunities for home monitoring of therapy, may suggest poor usage and may prompt clinicians to identify barriers to treatment, and similarly, patterns of use may provide important information about users’ beliefs and treatment choices, both positive and negative. As with continuous positive airway pressure therapy for sleep apnoea,31 demonstrating sufficient LVR use could be linked to funding of LVR therapy, potentially providing a commercialisation pathway for device development.

Conclusion

The LVR counter has been tested in both the laboratory and community settings and appears to be an effective usage monitor for both clinical and research purposes. Usage data that are more objective than the previous standard of self-report provide future research opportunities into the effectiveness of LVR treatment, prophylaxis and dose responsiveness.