Introduction

The timely diagnosis of chronic obstructive pulmonary disease (COPD) allows early adoption of interventions that have been shown to be beneficial in improving quality of life and health outcomes. Effective interventions include pneumococcal and influenza vaccination,13 referral to smoking cessation,46 pulmonary rehabilitation,79 and pharmacological therapy.10 Once diagnosed, the accurate classification of disease according to the degree of airflow obstruction predicts prognosis in COPD,11,12 allows the tailoring of pharmacological treatment,10 and can inform the prioritisation of resources to those who are in the greatest need.10 Conversely, a false positive diagnosis of COPD may lead to treatment that is either of no benefit or harmful. A false diagnosis of COPD may also cause the true underlying diagnosis to be missed, resulting in a failure to implement the appropriate effective therapy. For example, if asthma is misdiagnosed as COPD, the patient may be prescribed a long-acting bronchodilator without an inhaled corticosteroid (a treatment regimen that is contraindicated) rather than an inhaled corticosteroid.13

An important complicating factor in the diagnosis and classification of COPD is the highly heterogeneous nature of the condition, with severity of symptoms poorly correlated with degree of airflow obstruction.14 It is not surprising, then, that many diagnoses of COPD are not supported by spirometry15,16 and, for those that are, the agreement between recorded disease severity and degree of airflow obstruction can be poor.17

Guidance is available to assist in the diagnosis and subsequent management of COPD, internationally through the GOLD initiative18 and within England and Wales from the National Institute for Health and Care Excellence (NICE).10,14 Within both NICE and GOLD guidance the diagnosis of COPD and its severity classification is based on the degree of airflow obstruction as measured by spirometry. However, conducting spirometry is not straightforward,19,20 and can be of poor quality even with training.17,21 It has therefore been suggested that GPs should be provided with a spirometry service (in which spirometry is performed by a dedicated trained individual) rather than spirometers.2224

The primary aim of this study was to assess the accuracy of COPD diagnosis and the accuracy of the classification of airway obstruction, both in primary care and in a specialist respiratory care setting. A secondary aim was to compare the diagnosis and grade of airway obstruction at the time of referral from primary to specialist care with that subsequently found on assessment.

Our study took place in Rotherham, a mixed urban and rural district in the north of England with a population of approximately 250,000. Primary medical care in Rotherham is provided by 36 general practitioner (GP) practices, and secondary care is provided in a single large district general hospital. Rotherham also has a specialist nurse-led respiratory care centre, ‘BreathingSpace’. The unit is led by a respiratory nurse consultant and has a team of nursing, physiotherapy, and occupational therapy staff providing outpatient assessment, diagnosis, and treatment for those with COPD and other chronic respiratory conditions. Clinical and spirometric assessment for new referrals is performed by respiratory nurse specialists. The unit also has 15 inpatient beds for the care of patients with acute exacerbations of cardiorespiratory conditions. The aim of the BreathingSpace outpatient service is to ensure that patients in Rotherham with respiratory conditions are accurately diagnosed and optimally managed. Patients are primarily referred to the outpatient service for pulmonary rehabilitation, confirmation of diagnosis, management of symptoms, and medication review.

Methods

Data collection

Data were collected on all outpatient referrals from primary care to BreathingSpace. Medical record data are held by BreathingSpace in an electronic clinical system. Referrals to BreathingSpace are made using a standard form, and this information is uploaded to the clinical system at the time of referral. The details of each clinical encounter within BreathingSpace are also recorded on the system.

For the purposes of this study, an anonymised dataset was extracted from the clinical system and checked for completeness and accuracy by BreathingSpace staff. Data were extracted for all individuals who were referred from primary care to BreathingSpace for outpatient assessment, and who had their initial consultation at the centre between 1 May 2007 and 31 May 2010. Patients who attended BreathingSpace for reasons unrelated to COPD (i.e. those patients who were not suspected of having COPD on referral and who were not subsequently found to have COPD) were excluded, as were patients who did not attend their BreathingSpace appointment. If a patient was referred more than once, only the first referral was selected.

The following fields were included in the dataset: age, sex, the Index of Multiple Deprivation (IMD2010) measure of socioeconomic deprivation linked to the patient's postcode,25 date of referral, who the referrer was (either practice nurse or GP), the reason(s) for referral, any pre-referral diagnoses, the severity of airway obstruction stated in the referral (if COPD diagnosed), the most recent spirometry result stated in the referral (obtained from a spirometer tracing if this was included, otherwise as entered on the referral form), the date of initial assessment at BreathingSpace, the BreathingSpace spirometry result, any diagnoses made at BreathingSpace, and the grade of airflow obstruction based on the BreathingSpace assessment.

Definition of categories for reason for referral

We defined the following categories for reason for referral: assessment for pulmonary rehabilitation, symptom management, medication review, oxygen assessment, diagnostic assessment, patient request, education, and other. We recorded the numbers of referrals in each category. Some patients had more than one reason for referral recorded.

Study case definition for COPD and classification of airflow obstruction

The diagnosis of COPD and classification of airflow obstruction was based on the 2004 NICE Clinical Guideline (CG012) since this was the guidance that clinicians in Rotherham were expected to follow during the study period.14 Under this guidance, a diagnosis of COPD was made if the pre-bronchodilator forced expiratory volume in one second (FEV1) was <80% predicted and if the pre-bronchodilator FEV1/forced vital capacity (FVC) ratio was <0.7. Airflow obstruction was then classified as mild if the FEV1 was ≥50% predicted, moderate if the FEV1 was ≥30% and <50% predicted, and severe if the FEV1 was <30% predicted. We refer to this as the ‘grade’ rather than ‘severity’ of airflow obstruction to avoid any confusion with the severity of disease as experienced by the individual.

Note that the 2004 NICE Clinical Guideline CG012 was replaced by NICE Clinical Guideline CG101 in June 2010. Airflow obstruction is classified differently in the later guidance; hence, we included in our study only patients seen in BreathingSpace up to the end of May 2010.

Diagnostic accuracy in primary care: is each diagnosis of COPD and grade of airway obstruction consistent with the primary care spirometry?

We calculated the proportion of those referred with a primary care diagnosis of COPD whose primary care spirometry results were not consistent with COPD. We explored the association between diagnostic accuracy for COPD and a range of potential explanatory variables (patient's age, patient's sex, deprivation score linked to patient's postcode, referrer's profession, and date of referral) in a multivariate logistic regression model.

We assessed the agreement between the grade of airway obstruction as reported on the referral form and the grade of airflow obstruction based on the spirometry reported in the referral form. As our measure of agreement we used Cohen's weighted kappa coefficient.26 We also explored the association between airway obstruction grade agreement as a binary variable (i.e. agreement between stated grade and spirometry versus disagreement) and a range of potential explanatory variables (patient's age, patient's sex, deprivation score linked to patient's postcode, referrer's profession, and date of referral) in a multivariate logistic regression model.

Diagnostic accuracy in specialist care: is each diagnosis of COPD and grade of airway obstruction consistent with the specialist care spirometry?

We calculated the proportion of those diagnosed with COPD by BreathingSpace whose results from the spirometry conducted at BreathingSpace were not consistent with COPD. We explored the association between diagnostic accuracy for COPD and a range of potential explanatory variables (patient's age, patient's sex, deprivation score linked to patient's postcode, and date of assessment) in a multivariate logistic regression model.

We assessed the agreement between the grade of airflow obstruction recorded at the BreathingSpace assessment and the grade of airway obstruction based on the assessment spirometry conducted at BreathingSpace. We explored the association between agreement and the following potential explanatory variables: patient's age, patient's sex, deprivation score linked to patient's postcode, and date of assessment.

We used χ2 analyses to test for the difference between primary care and BreathingSpace in the proportions of patients misdiagnosed, and for the difference in the proportions misclassified with respect to grade of airway obstruction.

Comparison of the referral diagnosis and grade of airway obstruction with that found on specialist care assessment: are primary and specialist care findings consistent?

We calculated the proportion of patients with a primary care diagnosis of COPD whose spirometry conducted at BreathingSpace was not consistent with COPD. We explored the association between consistency (versus inconsistency) and a range of potential explanatory variables (patient's age, patient's sex, deprivation score linked to patient's postcode, referrer's profession, and date of referral) in a multivariate logistic regression model.

We compared the grade of airway obstruction stated on the referral form with that found on spirometric assessment at BreathingSpace. We explored the association between agreement and the following potential explanatory variables: patient's age, patient's sex, deprivation score linked to patient's postcode, referrer's profession, and length of time between referral and assessment.

We declared statistical significance at the conventional level of 5%. We considered values of Cohen's weighted kappa <0.4 as indicating poor agreement, values of 0.4–0.75 as indicating moderate to good agreement, and values >0.75 as indicating very good agreement.27 All analyses were conducted in R 2.15.3.28

Ethics approval

The Rotherham Research Alliance at the Rotherham NHS Foundation Trust reviewed the protocol of this study and deemed it as service audit not requiring ethical approval. The project was registered as a clinical audit at NHS Rotherham.

Results

A total of 1,205 patients met the inclusion criteria for the study. The mean (SD) age of the patients was 68 (10.2) years and just over half were male (630/1,205; 52%). Table 1 reports the reasons for referral. The majority of referrals were for assessment for pulmonary rehabilitation.

Table 1 Reason for referral to BretthingSpace

Diagnostic accuracy in primary care

COPD was listed as a primary care diagnosis in 1,044 (87%) of the 1,205 patients in the study. In 211 of these 1,044 cases (20.0%, 95% CI 17.9% to 22.8%) the spirometry result reported on the referral form was inconsistent with a diagnosis of COPD. In a multivariate logistic regression model, a spirometry-compatible diagnosis of COPD was not significantly associated with patient's gender, patient's age, postcode linked deprivation score, referrer's profession, or date of referral.

Table 2 reports the cross-classification between the grade of airflow obstruction as stated on the referral form and the grade based on the spirometry reported in the referral form. In 348 of the 1,205 referrals, both these pieces of information were available. There was only moderate agreement between stated grade and that based on the primary care recorded spirometry (43.4% of patients were misclassified; Cohen's weighted kappa 0.44, 95% CI 0.32 to 0.56). In a multivariate logistic regression model, referral by practice nurse (versus by doctor) was positively associated with agreement versus non-agreement (OR 1.85, 95% CI 1.33 to 2.57). There was no significant association with patient's gender, age, postcode linked deprivation score, or date of referral.

Table 2 Numbers of patients cross-categorised by airway obstruction grade as stated on the referral form and that based on the spirometry reported in the referral form

Diagnostic accuracy in specialist care

COPD was diagnosed by the specialist centre clinicians in 993 (82%) of the 1,205 patients in the study. Of these 993 cases, 65 (6.5%, 95% CI 5.2% to 8.3%) had a spirometry result that was inconsistent with a diagnosis of COPD. In a multivariate logistic regression model, a spirometry-compatible diagnosis of COPD was more likely in males than in females (OR 2.03, 95% CI 1.22 to 3.46), but was not significantly associated with patient's age, postcode linked deprivation score, or date of referral.

Table 3 reports the cross-classification between the grade of airflow obstruction recorded by BreathingSpace and the grade based on the assessment spirometry conducted at BreathingSpace. In 1,016 of the 1,205 referrals, both these pieces of information were available. There was very good agreement between the grade in the BreathingSpace record and that based on the BreathingSpace assessment spirometry (9.3% of patients were misclassified; Cohen's weighted kappa 0.88, 95% CI 0.81 to 0.96). A multivariate logistic regression analysis suggests that males are more likely to be correctly classified than females (OR 1.40, 95% CI 1.01 to 1.93). There was no significant association between agreement and patient's age, postcode linked deprivation score, or date of referral.

Table 3 Numbers of patients cross-categorised by airway obstruction grade as stated in the BreathingSpace record and that based on the spirometry conducted at BreathingSpace

The proportion of patients who had a diagnosis of COPD that was not supported by spirometry was significantly greater in primary care (20.2%) than at BreathingSpace (6.5%; χ2=80.0, p<0.0001). The proportion of patients misclassified with respect to grade of airway obstruction was also significantly greater in primary care (43.4%) than at BreathingSpace (9.3%; χ2=202.7, p<0.0001).

Comparison of diagnosis and airway obstruction grade between primary and specialist care

Of the 1,044 patients with a primary care diagnosis of COPD, 197 (18.9%, 95% CI 16.6% to 21.4%) had spirometry results at BreathingSpace that were inconsistent with a diagnosis of COPD. The most common diagnosis that was subsequently made on assessment at BreathingSpace was asthma (86 patients). Other diagnoses (with five or more patients) were bronchiectasis (12 patients), restrictive lung disease, (13 patients), and non-obstructive emphysema (5 patients). Thirty-three patients were found to have no respiratory disease.

In a multivariate logistic regression model, the following factors were associated with agreement between the primary care diagnosis of COPD and the BreathingSpace spirometry: the patient being male (OR 1.50, 95% CI 1.09 to 2.06) and the referral being made by a practice nurse (OR 1.65, 95% CI 1.18 to 2.33). There was no association with patient's age, postcode linked deprivation score, or length of time between referral and assessment.

Table 4 reports the cross-classification between the grade of airflow obstruction as stated on the referral form and the grade based on the assessment spirometry conducted at BreathingSpace. In 448 of the 1,205 referrals, both these pieces of information were available. The grade of airway obstruction found on spirometric assessment at BreathingSpace was different from that stated on the referral in 54.0% of patients (Cohen's weighted kappa 0.26, 95% CI 0.16 to 0.36). In a multivariate logistic regression model, referral by practice nurse (versus by doctor) was positively associated with agreement versus non-agreement (OR 2.20, 95% CI 1.61 to 3.00). There was no significant association with patient's gender, age, postcode linked deprivation score, or length of time between referral and assessment.

Table 4 Numbers of patients cross-categorised by airway obstruction grade as stated on the referral form and that based on the spirometry conducted at BreathingSpace

Discussion

Main findings

We found that diagnostic accuracy and accuracy of classification of airway obstruction grade was better in specialist respiratory care than in primary care. The grade of airflow obstruction found on assessment in the specialist centre did not agree with that stated on the referral form in over half of cases.

Strengths and limitations of this study

As far as we are aware, our study is the first to compare diagnostic accuracy for COPD between primary and specialist care. A key strength of the study is the large sample size, which represents approximately 20% of the known cases of COPD in Rotherham.29 The study does, however, have a number of limitations.

Rotherham represents only a single health administrative area in England and we should therefore be cautious in generalising the results more widely. In particular, we cannot be sure that our findings regarding BreathingSpace, a specialist respiratory centre that is somewhat unusual in being nurse-led, apply to specialist respiratory care settings in general.

We must be careful in attributing the differences we report between primary care and BreathingSpace to differences in the care provided. Case mix may also have differed in ways that could have impacted upon the outcomes that we measured. We did not, for example, record whether a patient was experiencing an exacerbation at the time of spirometric assessment (either in primary care or BreathingSpace). If there were differences in the proportion of patients who were experiencing an exacerbation (or who were unwell for other reasons) at the time of assessment between primary care and BreathingSpace, then this could explain some of the differences we have seen.

The study took place between 2007 and 2010, during the period in which the NICE 2004 guidance was in place. Practice is likely to have changed since then, in part due to the updated guidance issued by NICE in 2010. We do not know if the accuracy of diagnosis or of airway obstruction classification is better or worse under the new guidance.

As with any cross-sectional study, we cannot infer causation. Specifically, we do not know whether the associations that we report in the regression analyses are causal or related to unobserved confounding factors.

Although the grade of airflow obstruction can be determined using spirometry, spirometry alone does not give an adequate assessment of the disease severity experienced by an individual.10 To fully understand the impact of COPD on an individual, it is necessary to assess not only airflow obstruction but also symptoms, exercise capacity, risk of exacerbation, and degree of co-morbidity.30 It is possible that, in some instances, clinicians used the terminology ‘mild’, ‘moderate’, or ‘severe’ to refer to the severity of disease (even though no such classification exists within the NICE guidance) rather than the grade of airflow obstruction. This could account for some of the discrepancy seen in both primary and specialist care. An alternative would be to use composite scores such as the Dyspnoea, Obstruction, Smoking, Exacerbation (DOSE) or Body mass index, Obstruction, Dyspnoea, Exercise (BODE) indices.31,32 These can help in the assessment of severity and prognosis,3133 but their validity across the wide range of routine clinical settings is unknown.30

Our study population was drawn from those who were either diagnosed with COPD or who were suspected of having COPD. We therefore could not assess the degree to which COPD is underdiagnosed in the general population, although we know that this is likely to be significant.34,35 The number of patients in Rotherham who have a diagnosis of COPD recorded on their primary care record can be determined from the Quality and Outcomes Framework (QOF) dataset. The QOF is a pay-for-performance scheme for UK general medical practitioners which requires practices to keep patient registers for a range of chronic diseases including COPD. The total number of cases recorded in the QOF dataset for Rotherham in March 2012 was 6,431.29 This is considerably lower than the estimated number of cases that would be expected based on Rotherham's distribution of age, sex, ethnicity, rurality, smoking status and deprivation, which is approximately 10,000.36 Rotherham is not unusual in this respect; the number of diagnosed cases of COPD in England is approximately half that expected.34 Underdiagnosis is of particular concern because suboptimal management of COPD may lead to reduced quality of life, poorer outcomes, and avoidable admission to hospital.

Interpretation of findings in relation to previously published work

Our findings regarding diagnostic accuracy are broadly consistent with those found previously. One recent study similar to ours found that, of 180 patients with a diagnosis of COPD who were referred to a specialist service in London, 35 (19.4%) had no evidence of COPD on assessment.37 The prevalence of inconsistent spirometric findings in those diagnosed with COPD has also been reported in Sweden where it was 15% (n=533),15 in two Australian studies where it was 31% (n=341) and 48% (n=445),16,38 in Austria where it was 49% (n=68)39 and in two other UK studies where it was 32% (n=88) and 25% (n=189).40,41 With regard to the accuracy of classification of airflow obstruction grade, an audit of the quality of spirometry in the Rotherham COPD population found only moderate agreement between the description of severity in the medical record and that based on spirometry (Cohen's kappa 0.34, 95% CI 0.30 to 0.38).17

Of those patients with a primary care referral diagnosis of COPD who were subsequently found not to have COPD, a significant proportion had a diagnosis of asthma. This highlights the difficulty of differentiating between these two conditions in some patients.42 For example, in an elderly smoker who presents with breathlessness, wheeze and cough, there may be a tendency to diagnose COPD on the basis of the history, even in the presence of reversibility of airway obstruction.

When we compared the grade of airflow obstruction recorded on the referral form and that based on the BreathingSpace spirometry, we found that in only 46% of cases was the grade unchanged. This may well reflect the natural history of the disease as much as any problem of misclassification. The degree of airflow obstruction in COPD changes over time due, for example, to worsening of disease (either acutely during an exacerbation or more gradually over time) or to improvement in disease after exacerbation. Differences in grade of airflow obstruction between primary care assessment and BreathingSpace assessment may also be explained by changes in smoking status, treatment regimen, or degree of co-morbidity.

Implications for research, policy, and practice

The findings suggest that, in patients referred to specialist care, the primary care diagnosis of COPD and the classification of airflow obstruction should always be reviewed. Equally importantly, our findings highlight the need for good access to education for all healthcare professionals who have responsibility for the management of people with respiratory disease — in particular to education and training that relates to the interpretation of spirometry.

Diagnosis of COPD and its severity classification remain problematic. There is still no absolutely clear guidance on diagnosis and severity classification, even in the updated NICE guidance published in 2010.10 We suspect that this contributes to the problems of diagnostic inaccuracy that we have observed, and we would urge guideline producers to address this issue.

We found that the grade of airflow obstruction was more likely to be accurately recorded in referrals from practice nurses than in referrals from GPs. This may reflect the central role of nurses in managing the routine care of patients with chronic disease, a better familiarity with the interpretation of spirometry, more time, or perhaps a greater propensity to follow guidance strictly. Unfortunately, our study was not designed to determine why nurses were more accurate in reporting than doctors. There is, however, growing evidence to suggest that nurses represent an appropriate resource to deliver care to people with COPD throughout the whole of the disease pathway.43 Further research is needed to determine the relative effectiveness and cost-effectiveness of nurse-led versus GP-led care in the context of COPD.

Conclusions

Our results suggest that a proportion of patients diagnosed with COPD in primary care do not have COPD. Misdiagnosis is less common in specialist care. Misclassification of the grade of airflow obstruction is common in primary care and uncommon in specialist respiratory care.