Article Text

Validation of the STOP-Bang questionnaire as a screening tool for obstructive sleep apnoea in patients with cardiovascular risk factors: a systematic review and meta-analysis
  1. Mark Hwang1,
  2. Kevin Zhang2,
  3. Mahesh Nagappa3,
  4. Aparna Saripella4,
  5. Marina Englesakis5 and
  6. Frances Chung6
  1. 1Faculty of Medicine, Medicine, The University of British Columbia, Vancouver, British Columbia, Canada
  2. 2Faculty of Medicine, Medicine, University of Toronto, Toronto, Ontario, Canada
  3. 3Department of Anaesthesia & Perioperative Medicine, London Health Sciences Centre, London, Ontario, Canada
  4. 4Department of Anaesthesia and Pain Medicine, University Health Network, Toronto, Ontario, Canada
  5. 5Library and Information Services, University Health Network, Toronto, Ontario, Canada
  6. 6Department of Anaesthesia and Pain Medicine, Toronto Western Hospital, Toronto, Ontario, Canada
  1. Correspondence to Dr Frances Chung; Frances.Chung{at}uhn.ca

Abstract

Introduction Obstructive sleep apnoea (OSA) is highly prevalent in patients with cardiovascular risk factors and is associated with increased morbidity and mortality. This review presents the predictive parameters of the STOP-Bang questionnaire as a screening tool for OSA in this population.

Methods A search of databases was performed. The inclusion criteria were: (1) use of the STOP-Bang questionnaire to screen for OSA in adults (>18 years) with cardiovascular risk factors; (2) polysomnography or home sleep apnoea testing performed as a reference standard; (3) OSA defined by either Apnoea–Hypopnoea Index (AHI) or Respiratory Disturbance Index; and (4) data on predictive parameters of the STOP-Bang questionnaire. A random-effects model was used to obtain pooled predictive parameters of the STOP-Bang questionnaire.

Results The literature search resulted in 3888 articles, of which 9 papers met the inclusion criteria, involving 1894 patients. The average age of the included patients was 58±13 years with body mass index (BMI) of 30±6 kg/m2, and 64% were male. The STOP-Bang questionnaire has a sensitivity of 89.1%, 90.7% and 93.9% to screen for all (AHI ≥5), moderate-to-severe (AHI ≥15) and severe (AHI≥30) OSA, respectively. The specificity was 32.3%, 22.5% and 18.3% and the area under the curve (AUC) was 0.86, 0.65 and 0.52 for all, moderate-to-severe and severe OSA, respectively.

Conclusion The STOP-Bang questionnaire is an effective tool to screen for OSA (AHI≥5) with AUC of 0.86 in patients with cardiovascular risk factors.

  • sleep apnoea
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Key messages

What is the key question?

  • Is the STOP-Bang questionnaire a valid screening tool for obstructive sleep apnoea in patients with cardiovascular risk factors?

What is the bottom line?

  • The STOP-Bang questionnaire is a valid screening tool with high sensitivity to detect obstructive sleep apnoea in patients with cardiovascular risk factors.

Why read on?

  • This is the first study to review the current evidence on the use of the STOP-Bang questionnaire in patients with cardiovascular risk factors.

Introduction

Obstructive sleep apnoea (OSA) is a common sleep-related breathing disorder, characterised by frequent apnoea and hypopnea episodes.1 The prevalence of OSA is higher among individuals with a high body mass index (BMI), increased age, narrow airway, sedentary lifestyle and persons of African, Native and Hispanic descent.1 2 However, it is estimated that up to 80% of patients with moderate-to-severe OSA remain undiagnosed, leading to poor quality of life, increased healthcare utilisation, and exacerbation of comorbid conditions.3–5 The gold standard for diagnosis of OSA is polysomnography (PSG), which is costly and requires overnight observation at a sleep laboratory.6 Given the high prevalence of OSA in the general population and limited resources for PSG, a number of clinical screening tools have been developed to help prioritise high-risk patients for diagnosis and treatment.

The STOP-Bang questionnaire is an easy-to-administer OSA screening tool that has previously been validated in the sleep clinic and surgical populations.7–9 The questionnaire includes four binary (STOP: snoring, tiredness, observed apnoea and high blood pressure) and four demographic questions (BANG: BMI, age, neck circumference and gender).7 Originally validated in the surgical population, the STOP-Bang questionnaire reported a sensitivity of 83.9%, 92.9% and 100% to screen for all OSA (Apnoea–Hypopnea Index (AHI) ≥5), moderate-to-severe OSA (AHI ≥15) and severe OSA (AHI ≥30), respectively.7 Worldwide, the STOP-Bang questionnaire has been used extensively in preoperative clinics,7 10 11 various specialist clinics12 and in the general population.13

OSA has been associated with impaired endothelial function14 and various forms of cardiovascular disease, including hypertension, stroke, heart failure, atrial fibrillation and coronary artery disease.15 Given that patients with OSA and cardiovascular risk factors experience poor disease outcomes,16 there is a need for an easy-to-administer OSA screening tool to identify high-risk individuals for diagnosis and treatment as well as to reduce morbidity and mortality in this population. The purpose of this systematic review and meta-analysis is to evaluate whether the STOP-Bang questionnaire is a valid screening tool for the screening of OSA in patients with cardiovascular risk factors.

Methods

Study design and registration

The protocol of this study was registered in the International Prospective Register of Systematic Reviews (PROSPERO; registration CRD42020197390). We followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guideline.17

Literature search strategy

We searched for articles published in the following electronic databases: MEDLINE (Ovid), MEDLINE In-Process/ePubs, Embase, EmCare Nursing, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, PsycINFO, Web of Science (Clarivate), Scopus (Elsevier), CINAHL with full text (EBSCOhost) and Journals@Ovid using search strategy that was designed for each database by an information specialist (ME). All queries started from 2008 when the STOP-Bang questionnaire was first published and our search ended in March 2020. Only English abstracts were included in the search. No other limits were applied. The search strategy included the following free-text terms: ‘stop-bang’, ‘stopbang’. A manual citation search of Google Scholar and PubMed was conducted to identify missed articles. Continued literature surveillance was done through August 2020.

Study selection and data management

Two reviewers (MH, AS) independently conducted title and abstract screening using Rayyan. MH and KZ conducted full-text evaluation, data extraction and risk of bias assessment. Full-text articles were selected according to the following inclusion criteria: (1) the study assessed the STOP-Bang questionnaire as a screening tool for OSA in adults with cardiovascular risk factors; (2) PSG or home sleep apnoea testing was performed to confirm OSA diagnosis; (3) OSA was defined by an AHI or Respiratory Disturbance Index cut-offs ≥5, ≥15 and ≥30 events per hour; and (4) data on predictive parameters of the STOP-Bang questionnaire were provided. We defined cardiovascular risk factors as diabetes mellitus, hypertension, hyperlipidaemia, obesity, heart disease and cerebrovascular disease. In addition, disorders for which cardiovascular risk factors are major risk factors or part of the disease process were also included. All study subjects had at least one of the above risk factors to be included. Data from included studies were extracted using standardised data collection forms. Any disagreements between the reviewers were resolved by a third reviewer (MN). The data extraction process was managed using Microsoft Excel 2016.

Assessment of methodological quality

The internal and external validity of the included studies were assessed by the reviewers (MH, KZ) independently using the criteria coded according to the Cochrane Screening and Diagnostic Tests Methods Group.18 The reviewers met to discuss the results of their appraisal and any outstanding disagreements were resolved by a third reviewer (MN). The internal validity included the following criteria: valid reference standard, definition of disease, blind execution of index and reference tests, interpretation of index test independent of clinical information and study design. The external validity included the following criteria: spectrum of disease, clinical setting, previous screening or referral filter, demographic information, explicit cut-off of index test, percentage of missing participants, missing data management and participant selection for reference test. In addition, the reviewers graded the quality of each included study using the QUADAS tool with a score range from 0 to 14.19

Statistical analysis

For each of the included studies, the paired and unpaired predictive parameters were calculated using 2×2 contingency tables. A bivariate random-effects model was used to obtain pooled predictive parameters (sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), diagnostic OR and area under the curve (AUC)) to assess the validity of the STOP-Bang questionnaire for different AHI cut-offs: AHI ≥5, AHI ≥15 and AHI ≥30 events per hour. The meta-analysis was carried out with Review Manager V.5.4 and MetaDisc V.1.4.

Results

The literature search resulted in 3888 articles (figure 1). After screening titles and abstracts, 2279 studies were excluded because they did not meet the predetermined inclusion criteria. Of the remaining 17 studies, 8 full-text articles were excluded20–27 ; the reasons for exclusion are listed in online supplemental table S1. Nine studies met the inclusion criteria and were included in the review.28–36 The included studies encompassed 1894 patients and were conducted in six different countries: Australia,31 Canada,30 36 China,33–35 Czech Republic,29 Germany,32 and Italy.28

Figure 1

PRISMA study flow. AHI, apnoea-hypopnea index; OSA, obstructive sleep apnoea; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analyses.

The demographics of the included studies are summarised in table 1. The average age of patients with cardiovascular risk factors was 58±13 years with BMI of 30±6 kg/m2, and 64% were male. The characteristics of the included studies are summarised in table 2. All included studies were of prospective design, except one study which was retrospective.35 Six studies28–30 33 35 36 used AHI cut-off of ≥5, one31 used Respiratory Event Index ≥5 and two32 34 used AHI ≥15 to define OSA. The prevalence of OSA ranged from 38% to 96%. The cardiovascular risk factors explored in the studies included diabetes mellitus29 31–33 (n=4), stroke32 34 35 (n=3), heart disease30–32 (n=3), hypertension31 32 (n=2), retinal vein occlusion36 (n=1) and non-alcoholic fatty liver disease28 (n=1). Retinal vein occlusion was included for its close association with cardiovascular risk factors,37 38 which contribute directly to the pathogenesis.39 40 Similarly, non-alcoholic fatty liver disease was included as it is highly associated with cardiovascular risk factors41–43 in a bidirectional manner.44 Among the population with cardiovascular risk factors, six28–31 33 35 (n=1680), eight29–36 (n=1844) and five studies29–31 33 35 (n=1630) were included for meta-analysis at the AHI cut-offs of ≥5, ≥15, ≥30, respectively (figures 2 and 3, online supplemental figure S2).

Figure 2

Forest plots for pooled sensitivity and specificity of STOP-Bang questionnaire for various OSA severities in patients with cardiovascular risk factors. Values are presented as means with 95% CI in parentheses. AHI, Apnoea–Hypopnea index; Bang, BMI, age, neck circumference and gender; OSA, obstructive sleep apnoea; STOP, snoring, tiredness, observed apnoea and high blood pressure.

Figure 3

Forest plot for pooled diagnostic OR for various OSA severities for patients with cardiovascular risk factors. Values are presented as means with 95% CI in parentheses. AHI, Apnoea–Hypopnea index; OSA, obstructive sleep apnoea.

Table 1

Demographic data of patients using STOP-Bang questionnaire

Table 2

The characteristics of the included studies (n=9)

Methodological quality of the included studies

The QUADAS scores of included studies ranged from 11 to 14, indicating high quality studies with low risk of bias (table 1). All included studies used laboratory PSG or Home Sleep Apnoea Testing (HSAT) as a reference test to verify the accuracy of the STOP-Bang questionnaire, confirming internal validity (table 2). For validation purposes, four studies33–36 (44%) used PSG, while five28–32 (56%) used HSAT. Although PSG is the gold standard diagnostic modality, no significant differences were noted between studies that used PSG and HSAT with regard to OSA prevalence (table 2) and predictive parameters of the STOP-Bang questionnaire (figure 2). Blinding to the index and reference tests was done in four studies30 31 33 36 (44%), while the results of the STOP-Bang questionnaire were interpreted independent of clinical information in three studies30 31 33 (33%, online supplemental table S3). The appraisal of included studies based on criteria for external validity are summarised in online supplemental table S4. Eight studies28–31 33–36 (89%) clearly described the inclusion and exclusion criteria with the exception of one study.32 All nine studies provided enough information to identify the study setting, in addition to demographic data including age, gender, and BMI. All studies did not pre-screen for OSA prior to the application of the STOP-Bang questionnaire and all participants were invited or randomly selected for PSG or HSAT, suggesting an unbiased selection for the reference test.

Predictive parameters of the STOP-Bang questionnaire in patients with cardiovascular risk factors

The pooled predictive parameters of STOP-Bang score ≥3 to screen for OSA in patients with cardiovascular risk factors are presented in table 3, figures 2 and 3 and online supplemental figure S2. The prevalence of all OSA (AHI ≥5), moderate-to-severe OSA (AHI ≥15) and severe OSA (AHI ≥30) was 76%, 44% and 19%, respectively. The STOP-Bang questionnaire has a high pooled sensitivity of 89.1% (95% CI: 87.3% to 90.8%; I2=46.2%), 90.7% (95% CI: 88.5% to 92.6%; I2=84.4%),and 93.9% (95% CI: 90.1% to 96.3%; I2=10.1%) to screen for all (AHI ≥5), moderate-to-severe (AHI ≥15) and severe (AHI ≥30) OSA, respectively. We explored the heterogeneity contribution from these studies and found that in AHI ≥5, a study by Petta et al28 contributed to the maximum heterogeneity (online supplemental figure S5). Our sensitivity analysis by removing Petta et al28 found that heterogeneity decreased from 46.2% to 0% (online supplemental figure S5). Similar analysis was done for AHI ≥15, where we found that heterogeneity decreased from 84.4% to 65.3% after removing the Zeng et al.34 The pooled specificities were relatively low at 32.3% (95% CI: 27.8% to 37.2%; I2=75.2%), 22.5% (95% CI: 20.0% to 25.2%; I2=83.5%) and 18.3% (95% CI: 16.3% to 20.5%; I2=62.6%) for all, moderate-to-severe and severe OSA, respectively.

Table 3

Pooled predictive parameters of STOP-Bang score ≥3 to screen for OSA in patients with cardiovascular risk factors

The pooled positive predictive value (PPV) was highest at 80.7% (95% CI: 78.5% to 82.7%) to differentiate all OSA from non-OSA, with corresponding PPVs of 48.3% (95% CI: 45.8% to 50.8%) and 21.5% (95% CI: 19.4% to 23.8%) for moderate-to-severe and severe OSA, respectively (table 3). With a negative predictive value (NPV) of 92.7% (95% CI: 88.6% to 95.4%), the STOP-Bang questionnaire is most effective in ruling-out severe OSA. The corresponding NPVs are 48.3% (95% CI: 42.2% to 54.5%) and 75.2% (95% CI: 70.0% to 79.9%) for all OSA and moderate-to-severe OSA, respectively. The diagnostic odds ratio was 4.37 (2.83–6.75), 3.52 (2.60–4.77) and 3.72 (2.25–6.15) for all, moderate-to-severe and severe OSA, respectively. The AUC was 0.86, 0.65 and 0.52 for all, moderate-to-severe and severe OSA, respectively.

Predictive parameters of the various STOP-Bang scores

The predictive parameters of different STOP-Bang score cut-offs for all OSA (n=3045), moderate-to-severe OSA (n=3209) and severe OSA (n=2995) among patients with cardiovascular risk factors are summarised in online supplemental table S6. As the STOP-Bang score cut-off increased from 3 to 6, the sensitivity decreased from 89% to 14% for all OSA, 91% to 15% for moderate-to-severe OSA and 94% to 21% for severe OSA, respectively. For all OSA, moderate-to-severe OSA and severe OSA, there was an increase in specificity from 32% to 100%, 23% to 93% and 18% to 93%, respectively, when the cut-off increased from 3 to 6. The PPVs were high for all OSA at 81% for STOP-Bang cut-offs of ≥3 and highest at 100% for ≥6, respectively. The NPVs were highest for severe OSA at 93% for STOP-Bang cut-offs of ≥3 and high at 73% for cut-offs of ≥6, respectively.

Discussion

To date, this is the first meta-analysis of the validity of the STOP-Bang questionnaire in patients with cardiovascular risk factors. We found that the STOP-Bang questionnaire with a cut-off score ≥3 has excellent AUC at 0.86 in detecting OSA in patients with cardiovascular risk factors. Our findings are similar to those reported by previous reviews that validated the STOP-Bang questionnaire in the surgical and sleep clinic populations.7 8

The high false positive rate, and hence the low specificity, especially in detecting moderate-to-severe OSA, could be explained by the fact that some cardiovascular risk factor symptoms overlap with those of OSA. Heart failure and history of stroke, both of which showed high prevalence in our review, have previously been found to be accompanied by daytime fatigue and sleepiness.45–47 These symptoms could have increased the STOP-Bang score and thus resulted in false positives.

The sensitivity and specificity are important parameters for clinicians to consider when selecting a screening tool. The two are inversely proportional, meaning that high sensitivity usually comes at a cost to specificity and vice versa. In conditions for which the diagnostic test is costly, invasive or poses a risk to the patient, the screening tool should be highly specific, minimising false positives and the need to undergo the diagnostic test. In the context of OSA, the sensitivity of a screening tool is more important, as the diagnostic test (PSG or HSAT) is not associated with risk to the patient and is becoming increasingly affordable.48 49 In addition, missed OSA cases can incur substantial healthcare strain50–52 and indirect costs due to lost productivity and decreased quality-of-life,53–56 further substantiating the utility of a highly sensitive screening tool even at the cost of low specificity. The findings of our meta-analysis show that the STOP-Bang questionnaire fulfils this need as a valid screening tool for enhancing early detection of OSA in patients with cardiovascular risk factors.

Use of the STOP-Bang questionnaire in patients with cardiovascular risk factors

OSA is closely associated with various forms of cardiovascular disease,15 57–60 including hypertension, stroke, heart failure, atrial fibrillation, coronary artery disease and cerebrovascular disease, with increased risk of adverse outcomes.16 61 Given the high prevalence62 of undiagnosed and untreated OSA among patients with cardiovascular disease, a valid, accessible and easy-to-administer screening tool is crucial. The STOP-Bang questionnaire can be quickly administered as it involves a few simple questions and has been found to have a high response rate.7 63 The NPV of STOP-Bang score 0–2 allows us to safely exclude moderate-to-severe OSA in patients with cardiovascular risk factors. These characteristics, combined with the high sensitivity, make the STOP-Bang questionnaire a useful tool in identifying patients with cardiovascular risk factors who may be at high-risk for undiagnosed OSA to receive expedited diagnosis and treatment.

Although the current guidelines recommend continuous positive airway pressure (CPAP) treatment as initial therapy for patients with OSA,64 there is insufficient evidence on the efficacy of CPAP therapy in reducing all-cause or cardiovascular mortality and morbidity in patients with OSA and cardiovascular disease.65 66 One review found that CPAP therapy significantly improved sleep outcomes (AHI and Epworth sleepiness scale) and mental-component quality of life scores.65 As such, the current evidence is limited to demonstrating the efficacy of CPAP therapy to symptomatic improvements of OSA in patients with cardiovascular risk factors. Further research is needed to fully establish the impact of CPAP therapy and ultimately the effectiveness of screening tools such as the STOP-Bang questionnaire on improving clinical outcomes in cardiovascular patients with OSA.

Limitations

Our systematic review and meta-analysis has some limitations. Although all studies enroled patients without prescreening before the questionnaire was administered, the high prevalence could indicate some degree of selection bias. Second, there is significant between-study heterogeneity in sensitivity and specificity. One contributing factor to the observed variation could be the different validation devices HSAT or PSG among studies. Another factor for the variance may be the different geographic locations and clinical heterogeneity regarding the type of cardiovascular risk factor. Nevertheless, we used a random effects model, which is appropriate for a heterogenous sample. Third, the small number of included studies precluded the use of meta-regression analysis to establish the impact of potential confounders. Lastly, we only included studies with abstracts in the English language. Despite these limitations, our systematic review and meta-analysis provide a summary of the current literature on the STOP-Bang questionnaire and recommends its use as a screening tool in patients with OSA and cardiovascular risk factors.

Conclusions

In summary, this meta-analysis shows that the STOP-Bang questionnaire is a valid screening tool for detecting OSA in patients with cardiovascular risk factors. The high sensitivity and PPV of the STOP-Bang questionnaire enables risk stratification and early detection, facilitating the diagnosis and treatment of OSA.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors Study concept and design was performed by MH, MN and FC. Literature search was done by ME. Acquisition, analysis and interpretation of data were done by all the authors. Statistical analysis was performed by MH, KZ, MN and AS. Drafting of the manuscript was done by MH, KZ, MN and FC. All authors performed critical revision of the manuscript for important intellectual content. FC guarantees the integrity of the work.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.

  • Author note The institution where the work was performed: University Health Network, University of Toronto, MCL 2-405, 399 Bathurst Street, Toronto, ON, Canada, M5T2S8.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.