Work-related asthma (WRA)is the most common non-asbestos related occupational respiratory disease in the US and in most industrialised nations.^1–4 It accounts for about 25% of all reported occupational lung diseases in the UK⁵ and an estimated 15% of asthma cases in the US are work related.^4,6–8 In the US, the Sentinel Event Notification System for Occupational Risks (SENSOR) case-classification scheme for WRA encompasses both new-onset asthma and work-aggravated asthma.⁹ Work-aggravated asthma is defined as the aggravation or exacerbation of pre-existing asthma due to occupational exposures. Based on SENSOR data of four states from 1993 to 1995, 19.1% of WRA cases were in the aggravation category.⁹ These cases had similar adverse outcomes as the new-onset cases, such as persistent breathing problems and the need for emergency care.¹⁰

Questionnaires that gather information on symptoms and exposures are frequently used to assess WRA.¹¹ Questionnaires are inexpensive and feasible for large population-based studies, but patients must be able to clearly perceive their respiratory status for their responses to be useful. Unfortunately, asthma patients’ perceptions of dyspnoea are not always consistent with measured levels of air flow obstruction.¹² One study showed that 15% of people with asthma failed to be aware of any discomfort after a 20% reduction in their forced expiratory volume in one second.¹³ The inability of a patient to accurately perceive substantial changes in his or her lung function results in inaccurate reporting of symptoms due to either blunted perception or over perception.¹⁴ Therefore, self-reported symptoms could not be effective surrogates for more objective diagnostic tools in determining the contribution of exposures at work to the onset or exacerbation of asthma.

Serial measurements of peak expiratory flow (PEF) are an established objective aid in the diagnosis of WRA.^15,16 The test allows for simple, inexpensive measurement of lung function outside of a clinic or laboratory.^15,17–19 Serial PEF measurements have a high sensitivity and even higher specificity for WRA.^19–21 For example, when using specific inhalation challenges as the standard for occupational asthma, serial PEF measurements were found to have a sensitivity of 70–75% and a specificity of 94–100%.^20,21

The primary objective of this investigation was to validate work-related self-reported exacerbation of asthma using the findings from serial PEF measurements as the standard.

METHODS

The protocol for the study was reviewed and approved by the National Institute for Occupational Safety and Health (NIOSH) Human Subjects Review Board, and the Human Subjects Review Board of the contracted Health Maintenance organization (HMO), Fallon Clinic Research Department in Eastern and Central Massachusetts, Worcester, Massachusetts, USA. All information was de-identified before it was sent to NIOSH.

Participants

The current validation study was part of the Workplace Exacerbation of Asthma (WEA) Research Project, which was initiated by NIOSH in 2000 to investigate the frequency, causes and consequences of work-related exacerbation of asthma. NIOSH contracted with researchers at the Fallon Clinic in Massachusetts to conduct data collection. Detailed methods for the baseline study are presented elsewhere.²² In brief, both electronic and paper medical records were reviewed to identify adults aged 18–44 years with diagnosed asthma who had been enrolled in the Fallon Community Health Plan for at least 6 months.

From the 598 baseline study participants, candidates for the current validation study were identified. However, 192 baseline study participants were not invited to take part in the validation study because they were no longer enrolled in Fallon Community Health Plan. Research staff sent materials, including an invitation to participate and an explanation of the study, to the remaining 406 people.

Materials

Office spirometry was accomplished using an OMI Sensormedics 922 spirometer (Occupational Marketing, Houston, Texas, USA). Collection of serial PEF measurements was achieved through the use of ndd EasyOne portable spirometers (ndd Medical Technologies, Andover, Massachusetts, USA). Both the office and portable spirometers were calibrated to meet American Thoracic Society standards.²³ The portable devices were self-recording, capable of storing large numbers of spirometric measurements and included an alarm that sounded at regular intervals to remind participants to test themselves. In addition, participants were prompted at each trial to complete a daily log that recorded symptoms and medication use. Participants entered their answers by means of a keypad built into the side of the spirometer. All the information from the spirometer was date and time stamped. Information was also used from two other survey instruments: the baseline study questionnaire and a telephone questionnaire conducted after the serial PEF testing.

Serial spirometry

Before the start of serial testing, participants were asked to complete an office spirometry session, where they were given instruction on using the portable spirometers. Measurements of PEF were performed for 3 weeks at home and at work. The participants were asked to conduct at least five trials each day with at least three forced expiratory manoeuvres performed at each trial.^17,18,20,24 The best of the three measurements was used for analysis. On completion of both the office spirometry and at least 16 days of serial testing, the participants were given $200 as compensation for the time it took to complete these tests.

Self-reporting

Work-related self-reported exacerbation of asthma was assessed in two ways. First, symptoms and medication use concurrent with serial testing were gathered from the daily log. Questions included (1) In the last 2 h, did you have a cough attack, wheeze, chest tightness or shortness of breath? (2) In the last 2 h, did you use your fast-acting inhaler? Responses to each of these questions were compared between work days and non-work days. Participants with a greater percentage of positive responses on all work days than on all non-work days were classified as having work-related self-reported exacerbation of asthma. Two different operational definitions were derived, shown below.

(1) Reported symptoms more often on work days versus non-work days.

(2) Reported medication use more often on work days versus non-work days.

Second, symptoms and medication use were collected by a post-test telephone questionnaire that asked about the previous 3 weeks of testing. Those who reported that their symptoms or medication use varied by whether they were at or away from work were asked the following questions: (1) Did your asthma symptoms get better or worse away from work? (2) Did you use your inhaler or nebuliser more or less on work days? The following operational definitions were based on answers to the post-test questions:

(1) Asthma symptoms improved away from work.

(2) Medication use increased on work days.

Validation standard

The participants returned the portable spirometer to the HMO after completion of testing. Research staff entered the data into the Occupational Asthma System 2 (OASYS-2), which is a computer-assisted diagnostic aid that was developed to detect asthma–work relationships using serial PEF data.^25,26 The computer program generates plots of daily maximum, mean and minimum PEF values, and a work-relatedness score ranging from 1 to 4. The probability of WRA increases as the score increases, and a score of >2.5 has a sensitivity of 75% and a specificity of 94% when using specific inhalation challenge as the standard for occupational asthma.²⁰

Three researchers familiar with the interpretation of serial PEF measurements reviewed all the OASYS-2 records. These records were judged to have inadequate data for evaluating a work relationship if participants completed <2 weeks of ⩾4 trials a day, provided an insufficient number (ie, <2) of work-day/rest-day complexes, or had a respiratory infection during the testing period. Infections were reported to the field staff, who in turn, recorded the information and provided it to the researchers who analysed the data. A complex was defined as a series of at least three work days that was preceded and followed by at least two rest days, or at least two rest days preceded and followed by at least three work days.

The three reviewers independently judged the remaining OASYS-2 records to be either positive or negative for WEA based on serial PEF (WEA–PEF). Reviewers were unaware of any work-related respiratory problems that participants might have reported in the baseline and validation studies. The primary features considered when judging the relationship to work were the OASYS-2 score, lower PEF values while working and higher PEF values while away from work and increased diurnal variation on work days. Diurnal variation was defined as the difference between maximum PEF and minimum PEF divided by the mean PEF for the whole period. The category agreed upon by the majority of reviewers was used for analysis.

Data analysis

All statistical analyses were accomplished using SAS 9.1 software. The analysis included comparisons of demographic features and validity calculations. Data from the baseline study questionnaire were used for gender, age, race, smoking status and presence of atopy. Participants were considered positive for atopy if they had ever been told by a doctor that they had hay fever or nasal allergies. To test for statistical significance when making comparisons, the Student’s t test was used for continuous variables, and the continuity corrected χ² statistic and Fisher’s exact test were used for categorical variables. The Fisher’s exact test was used when expected cell counts were <5. A significance level of 0.05 was applied for all tests.

Validity measurements were calculated to characterise the ability of self-reported symptoms and medication use to identify who did and who did not have work-related disease, using WEA–PEF status as the standard. Contingency tables were created for single operational definitions and either–or/both–and combinations of the operational definitions (n = 66). Sensitivity, specificity and Youden’s Index were calculated for each table. Youden’s Index is the sum of the sensitivity and specificity (expressed as fractions) minus one.²⁷ When sensitivity and specificity are equal to one, Youden’s Index is zero, and the operational definition used was no better than random chance.²⁸

RESULTS

Of the 406 adults with asthma who were asked to participate in the validation study, 382 were employed at the time of testing. Figure 1 summarises the disposition of these 382 adults. In all, 178 of 382 (47%) participants completed office spirometry; the other 204 declined to participate or did not respond. Of those who completed office spirometry, 40 did not complete at least 2 weeks of serial spirometry testing, whereas 138 met the 2-week requirement. Forty-three of 138 did not have reviewable records due to an insufficient number of work days or too few consecutive work days (n = 28), an insufficient number of rest days (n = 6), skipped days mid-testing (n = 5) or the presence of a respiratory infection during testing (n = 4). In total, 95 of 382 (25%) participants completed serial spirometry testing and provided adequate data to be evaluated for WEA–PEF.

We compared demographic features among the 204 employed candidates for participation who refused or never responded to the invitation, the 83 who completed office spirometry but not serial testing and the 95 who completed both office and serial spirometry. There was little variation in gender, age, race or atopy by participation status (table 1). However, the two groups which completed office spirometry had a lower percentage of ever-smokers than the non-participants (p<0.05) and those groups that completed the serial spirometry had a higher percentage of moderate asthma. The 24 adults with asthma who were not invited due to unemployment were more likely to be female than those who were invited (p = 0.01), but were approximately of the same age. The 192 adults with asthma who were not invited owing to insurance compliance were similar to those who were invited with respect to gender and age (data not shown).

Upon evaluation of the OASYS-2 records, 13 of 95 (14%) participants were positive for WEA–PEF. Those with WEA–PEF were more likely to be female and less likely to be atopic, although these differences were not statistically significant (table 2). Also, age and smoking status did not vary by WEA–PEF status.

Twenty-five of 95 (26%) participants changed occupations from baseline to the time of testing. Three of these people were performing the same tasks at a different location. The prevalence of WEA–PEF was higher among those who changed employment (18%) compared to those who did not (12%), but the difference was not statistically significant (p = 0.49). The broad occupational category with the highest prevalence (23%) of WEA–PEF was service occupations, whereas the lowest prevalence (7%) was in the sales and office occupations (table 3). The highest prevalences by specific categories were 40% (2 of 5) for healthcare support (in service occupations) and 29% (4 of 14) for education, training, media and library (in management, professional and business occupations).

Fulfilment of the operational definitions of work-related symptoms and medication use was about three times more common when based on reports concurrent with the serial testing as compared with reports from the post-test telephone interview. Specifically, concurrent and post-test reports were 48% vs 14% for symptoms and 39% vs 12% for medication use, respectively.

Sensitivity, specificity and Youden’s Index were calculated for operational definitions based on both types of self-reported measures. Table 4 shows the single operational definitions, and also all combinations of definitions that had a Youden’s Index ⩾0.30 (top 1/5 of all the combinations).

None of the operational definitions for work-related self-reported exacerbation of asthma had very high values for both sensitivity and specificity (table 4). The highest specificity was observed for post-test symptoms (87%) and medication use (89%), but both had a very low sensitivity of 15%. The highest Youden’s Index for a single self-reported operational definition was 0.27 for concurrent medication use, with a sensitivity of 62% (8 of 13 cases identified) and a specificity of 65% (53 of 82 cases with serial PEF negative for WEA). When the operational definitions were expanded to include either concurrent or post-test medication use, the sensitivity increased to 69%, which was the highest level attained. Nine of the 13 true cases were identified, and the specificity (61%) was nearly the same as that observed for concurrent medication use by itself. A sensitivity of 69% was also achieved for the operational definition of both post-test symptoms and medication use or concurrent medication use.

DISCUSSION

Serial PEF testing by adults with asthma

Most study candidates did not provide useful serial spirometry data, even though they were offered money as compensation for their time. Only 138, or about one-third of the 382 eligible, completed at least 2 weeks of serial testing. In addition, 34 of 138 had data that could not be analysed by the OASYS-2 software owing to too few work or rest days, or too few consecutive work days. This last group was the result of part-time or non-traditional work schedules. Lack of compliance and inadequate data collection have been observed in studies limited to workers with suspected WRA.^30,31 The low compliance in this study suggests that serial PEF testing is of limited usefulness in population-based or quasi-population-based studies, at least when trying to determine work-relatedness. This problem could possibly be overcome by offering higher monetary incentives that exceed merely compensating participants for their time, and by initiating more frequent (perhaps daily) contact with the participants to keep them on track with the study.

The adults with asthma who completed serial testing were more likely to have moderate asthma rather than mild asthma and less likely to have ever smoked than those who completed neither office nor serial spirometry. However, neither characteristic was associated with a positive serial test result. At the same time, the adults with asthma who did complete the serial testing were very similar to those who did not with respect to gender, age, race and atopic status. Thus, the findings based on the 95 participants are probably representative of the entire group of candidates for participation. Moreover, we determined that 14% of the participants had WEA–PEF, which is similar to the prevalence estimate for WRA in the American Thoracic Society statement.⁸ Still, we cannot entirely rule out that selection bias could have occurred, especially given the low rate of compliance, and this could have biased the findings in either direction.

A major criticism of serial self-testing has been the potential for inaccurate recording of PEF measurements by participants.^24,32 In this study, participants used self-recording portable spirometers, making the recording of PEF measurements more accurate. A learning effect has also been reported while people are becoming accustomed to using the spirometers. The current participants were oriented to the portable spirometers in person, and the devices provided coaching messages each time they were used. These attributes helped to assure that the PEF readings were acceptable and accurately recorded.

Overall, this study shows that serial PEF measurements, once collected, can be used to determine work-related patterns of asthma. Serial spirometry is an inexpensive method of screening for WRA that is more objective than self-report and readily available in many countries. A drawback is that our standard has limitations with regard to measuring WRA, with a sensitivity of approximately 70–75% for occupational asthma when compared with specific inhalation challenge.^20,21 If the same limitation of serial spirometry applies to WEA, then our estimates of sensitivity and/or specificity for self-reports could be inaccurate.

Occupations with a higher prevalence of WEA

The highest prevalence of WEA–PEF was found in the occupation subcategory of healthcare support that includes nursing aides, home health aides, dental assistants and so on. In a review of SENSOR data from 1993 to 1997, exploring the association of WRA and cleaning products, nurse’s aides had the second highest prevalence of WRA (20%).³³ The other agents associated with WRA among healthcare workers were latex and indoor air pollution.³⁴ The second highest prevalence of WEA was found in the occupational subcategory of education, training, media and library. In a review of SENSOR data from Massachusetts (1993–2004), educational services industry had the second highest prevalence of WRA.³⁵

The value of self-reports in the identification of work-exacerbated asthma

In some other studies that have examined the value of self-reported work-related symptoms, the participants were workers with asthma seeking compensation.³⁶ Participants in this study were adults with asthma who were being treated in an HMO. In this setting, the false positives based on self-reports are unlikely to represent intentional falsification to gain compensation, and are more likely due to misperception of causation or the presence of work-related symptoms in the absence of significant changes in the PEF. False positives were common in this study—for example, of the 37 participants who fulfilled the operational definition based on concurrent medication use, 29 (78%) were negative for WEA based on PEF. Consistent with this observation, some researchers have suggested that work-related exacerbation of asthma is often characterised by changes in symptoms without changes in the underlying physiology or pathology of the disease.³⁷

The reports of symptoms and medication use that the participants recorded, concurrent with PEF self-testing, were indications of events that had occurred in the past 2 h. In contrast, the post-test questions about symptoms and medication use required the participants to recall what had occurred weeks earlier, and to determine whether there had been a difference between work days and non-work days. The approach based on concurrent reports had a higher sensitivity for detecting work-exacerbated asthma cases than the approach based on post-test reports (62% vs 15%, respectively).

As clinical evaluation for suspected work-exacerbated asthma is often limited to those who have self-identified based on their perceived symptom or medication use patterns, many people who would probably qualify for compensation are not being identified. This same problem would diminish the number of cases reported by treating doctors to surveillance systems. Although concurrent reports of medication use were more effective at identifying WEA–PEF than post-test reports, this approach still resulted in many false positives and false negatives.

CONCLUSIONS

Serial PEF measurements provided objective evidence for WEA but participation was poor in this cohort of adults with asthma. At the same time, those who completed the serial spirometry testing seemed to be representative of the potential participants, and 14% of them were determined to have WEA–PEF. With WEA–PEF status as the standard for WEA, self-reports of symptoms and medication use were inaccurate, resulting in both false positives and false negatives. If clinical evaluation for suspected WEA is limited to those who have self-identified, then many people with this condition who would potentially qualify for compensation are not being identified. The overall performance of self-reports in identifying work-related patterns of PEF among adults with asthma illuminates the need for further research in the area of development and validation of the questionnaire.