Article Text
Abstract
Background There is a lack of individualised prediction models for patients hospitalised with chronic obstructive pulmonary disease (COPD) for clinical practice. We developed and validated prediction models of severe exacerbations and readmissions in patients hospitalised for COPD exacerbation (SERCO).
Methods Data were obtained from the Acute Exacerbations of Chronic Obstructive Pulmonary Disease Inpatient Registry study (NCT02657525) in China. Cause-specific hazard models were used to estimate coefficients. C-statistic was used to evaluate the discrimination. Slope and intercept were used to evaluate the calibration and used for model adjustment. Models were validated internally by 10-fold cross-validation and externally using data from different regions. Risk-stratified scoring scales and nomograms were provided. The discrimination ability of the SERCO model was compared with the exacerbation history in the previous year.
Results Two sets with 2196 and 1869 patients from different geographical regions were used for model development and external validation. The 12-month severe exacerbations cumulative incidence rates were 11.55% (95% CI 10.06% to 13.16%) in development cohorts and 12.30% (95% CI 10.67% to 14.05%) in validation cohorts. The COPD-specific readmission incidence rates were 11.31% (95% CI 9.83% to 12.91%) and 12.26% (95% CI 10.63% to 14.02%), respectively. Demographic characteristics, medical history, comorbidities, drug usage, Global Initiative for Chronic Obstructive Lung Disease stage and interactions were included as predictors. C-indexes for severe exacerbations were 77.3 (95% CI 70.7 to 83.9), 76.5 (95% CI 72.6 to 80.4) and 74.7 (95% CI 71.2 to 78.2) at 1, 6 and 12 months. The corresponding values for readmissions were 77.1 (95% CI 70.1 to 84.0), 76.3 (95% CI 72.3 to 80.4) and 74.5 (95% CI 71.0 to 78.0). The SERCO model was consistently discriminative and accurate with C-indexes in the derivation and internal validation groups. In external validation, the C-indexes were relatively lower at 60–70 levels. The SERCO model discriminated outcomes better than prior severe exacerbation history. The slope and intercept after adjustment showed close agreement between predicted and observed risks. However, in external validation, the models may overestimate the risk in higher-risk groups. The model-driven risk groups showed significant disparities in prognosis.
Conclusion The SERCO model provides individual predictions for severe exacerbation and COPD-specific readmission risk, which enables identifying high-risk patients and implementing personalised preventive intervention for patients with COPD.
- COPD exacerbations
Data availability statement
Data are available on reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Acute exacerbations of chronic obstructive pulmonary disease (AECOPD) are frequent and significant causes of readmission and mortality. However, there is a lack of individualised prediction models for patients hospitalised with COPD for clinical application.
WHAT THIS STUDY ADDS
Among patients hospitalised for AECOPD, the severe exacerbations and readmissions in patients hospitalised for COPD exacerbation (SERCO) model shows good performance for predicting the risk of severe exacerbations and COPD-specific readmissions at 1, 6 and 12 months after discharge.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This study highlights that the SERCO model could be an easy-to-use tool to predict the individual risk of re-exacerbations for inpatients with AECOPD, which enables to identification of ‘high-risk’ patients and facilitates preventative intervention implementation.
Introduction
Acute exacerbations of chronic obstructive pulmonary disease (AECOPD) are frequent and significant causes of readmission and mortality. Approximately 63% of patients will readmit to the hospital for all causes within 1 year following hospitalisation for AECOPD, and more than 50% of them will die in 5 years.1 2 Due to the poor prognosis and high burden of AECOPD, reducing exacerbations and readmissions is the major goal of improving COPD care quality.3 4 Identification of the exacerbation risk will be a possible approach to identify those who may benefit most from additional care and apply personalised prevention treatments to reduce exacerbations.5 Prior exacerbation history is the currently widely used predictor of future exacerbations.6 7 However, even if COPD patients all have a previous history of exacerbations, heterogeneity of clinical phenotypes, and prognosis could be obvious. For example, certain patients are susceptible to frequent exacerbations (defined as two or more exacerbations per year) and have worse health status and morbidity than patients with less frequent exacerbations.8 Previous exacerbation history may be more suitable for population-based prediction rather than individualised prediction. Several systematic reviews have evaluated the prediction tools for the prognosis of COPD.9–12 Few existing prognostic models reported model validation and risk stratification5 9 13 14 and have been recommended by guidelines and applied clinically worldwide.7
The identification of ‘high-risk’ patients with severe exacerbations and readmissions will facilitate individual preventative intervention implementation and improve the care quality for patients with COPD. This study aimed to develop and validate integrated prediction models and stratified the risk for severe exacerbations and readmissions in patients hospitalised for COPD exacerbation (SERCO) during a 12-month period postdischarge including 1 month, 6 months and 12 months.
Methods
Participants and study design
The recommendations set by the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis statement were followed. The data were obtained from the Acute Exacerbations of Chronic Obstructive Pulmonary Disease Inpatient Registry (ACURE) study, a nationwide, multicentre, ongoing prospective cohort study in China (ClinicalTrials.gov identifier: NCT02657525).15 The enrolment of participants began on 1 September 2017, and the study expected to recruit 7600 patients with a 3-year follow-up. The ACURE study adopted a multistage, stratified and cluster sampling method to recruit hospitals from mainland China. Considering the participation willingness and treatment ability, 176 secondary and tertiary sites distributed across 29 provinces in China were finally selected by November 2021. In this study, we collected the data until 5 November 2021. Details of the ACURE study have been previously described.15
The eligibility criteria for the ACURE study were (1) aged ≥18 years, (2) confirmed or suspected hospitalisation for AECOPD, (3) not participating in other clinical trials or interventional studies and (4) signed consent for participation. Following the criteria in the guideline endorsed by the Global Initiative for Chronic Obstructive Lung Disease (GOLD) scientific committee, AECOPD is defined as spirometry-confirmed COPD (a postbronchodilator ratio of forced expiratory volume in 1 s (FEV1) to forced vital capacity (<0.70) with acute exacerbations of respiratory symptoms requiring additional treatment.7 Since patients with AECOPD were not suitable for spirometry, patients underwent spirometry before discharge.16 For those who were unable to complete spirometry, the results of previous pulmonary function tests from 6 months before hospitalisation or 30 days to 6 months after discharge were used to supplement the diagnosis of COPD. To prospectively develop the 12-month course prediction model, we additionally excluded the patients who lost to follow-up (without any follow-up record during the 12-month period after discharge).
Model development and validation
Due to significant geographical disparities in risk factors and prevalence of COPD,17 we divided the study dataset into development and external validation cohorts by geographical regions in which the patients were enrolled. Those from Northern, Northeast and Eastern China were assigned to the development cohort. Patients from Southern, Southwest, Northwest and Central China were assigned to the external validation cohort. The procedures of the study are shown in figure 1. The provinces in each region are detailed in online supplemental table S1.
Supplemental material
Outcomes and potential predictors
The primary outcomes of interest were severe exacerbations and COPD-specific readmissions. Severe exacerbations were defined as an acute episode of intensified symptoms that requires admission to emergency or hospitalisation.7 COPD-related death was considered as part of the outcomes in our study. COPD-specific readmissions were defined as severe exacerbations of COPD that required readmission to the hospital after discharge. To evaluate the time-depend risk of the outcomes, the date and severity of each exacerbation during the follow-up were recorded, and the 1-month, 6-month and 12-month incidence of outcomes was calculated.
Based on the existing knowledge and literature review, the prespecified potential predictors were considered if they were clinically relevant and available.11 The candidate predictors included demographic characteristics, COPD-related disease history, comorbidities and complications, treatment during hospitalisation and clinical characteristics. All the potential predictors are listed in online supplemental table S2.
Follow-up
All the participants were expected to complete the 12-month follow-up after discharge, within which the face-to-face visits were done at 1 month, 6 months and 12 months, and telephone interviews were done at 3 months and 9 months after discharge. Any event of interest, the date of event occurrence and other detailed information were recorded by trained physicians.
Patient and public involvement
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
Statistical analysis
Descriptive statistics were reported as frequencies and proportions for categorical variables, and median (IQR) or mean (SD) for continuous variables. The cumulative incidence and 95% CIs of severe exacerbations and COPD-specific readmissions were estimated using the cumulative incidence function (CIF)18 that simultaneously accounts for competing risks caused by death during the follow-up.
For model development, multivariable cause-specific hazard models19 that simultaneously account for competing risks were used to examine the association between candidate predictors (described above) with the 12-month severe exacerbation and readmission. We tested all the interactions among the potential predictors (shown in online supplemental table S3). All the interaction terms with p value <0.05 were then included in the full predictor model for selection. Together with the significant interactions, the candidate predictors were all included in a full, plausible, maximally complex model (shown in online supplemental table S4), and then considered both the clinical and the statistical magnitude of predictors to select a parsimonious model. We selected binary or multilevel categorical predictors associated with coefficients >0.1 or <−0.1 (it means HR>1.1 or <0.9) at p<0.05 for inclusion, and interactions and continuous variables were selected if associated with p<0.05. Then these were used to refit the final model. The final model included the data-driven selected variables and clinically significant predictors that physicians selected. Smoking was also included for its importance even though its p value was >0.05. For clinical application, predictors in the COPD-specific readmission model were the same as the severe exacerbation model.
We categorised postbronchodilator per cent predicted FEV1 (FEV1%predicted) by GOLD stages (a postbronchodilator FEV1%pred≥80%, 50% to <80%, 30% to <50% and<30% represented GOLD 1–4, respectively).7 Age, body mass index (BMI), frequency of hospitalisations for AECOPD in the past 12 months, modified British Medical Research Council (mMRC)20 and COPD Assessment Test (CAT) score21 were assessed as their original continuous scale. Other predictors were assessed as binary variables. In the complete cohort, there were 2.0% (81/4065) and 1.3% (52/4065) missing values of mMRC and CAT score, respectively. Missing values were imputed using the multiple imputation by chained equations method.
Coefficients of predictors were estimated in the final regression models and HRs with 95% CI were calculated. Combining with the observed datum risk, the predicted probabilities of outcomes were then calculated following the formula: p=1-R0exp ((β1×X1+β2×X2+……+βn×Xn)-(β1×X1m+β2×X2m+……+βn×Xnm)). In the formula, R0 stands for the observed datum risk of outcomes at an appointed time in the development cohort calculated by CIF, βn for the coefficient, Xn for the predictor, and Xnm for the mean value of the predictor in the development cohort. Depending on the consistent coefficient and time-specified datum risk, we developed the model predicting the 1-month, 6-month and 12-month probabilities of outcomes. For the easy-to-use purpose, the visual probabilities were estimated by nomogram.
The discrimination and calibration of the prediction models were evaluated. Discrimination was assessed by calculating the C-index. We also compared the discrimination of our models with prior severe exacerbation history in the past year6 by the area under the curve (AUC) of receiver operating characteristic (ROC) curves. Calibration was evaluated by slope and intercept, which define the linear relationship between predicted and actual probabilities. The slopes and intercepts in the internal validation cohort were then used to adjust the predicted probabilities following the formula: Padj=(1+intercept)-R0exp (((β1×X1+β2×X2+……+βn×Xn)-(β1×X1m+β2×X2m+……+βn×Xnm))×slope).22
Decision curves analysis was used to assess the clinical utility (net benefit).23 This analysis assesses the trade-off between the benefits of true positives and the potential harms that may arise from false positives across a range of threshold probabilities.
Both internal and external validation were done in this study. The 10-fold cross-over validation was performed to estimate the model performance. To assess external validity, the prediction accuracy of outcomes was determined in the external cohort by computing the C-index and calibration plots. According to the optimal cut-off value from the AUC analysis, the patients were stratified into low-risk (predicted 12-month risk<16%) and high-risk (predicted 12-month risk≥16%) groups by predicted individual probabilities. CIF was used to determine the incidence rate differences between the low-risk and high-risk groups. The models’ sensitivity, specificity, positive and negative predictive values were then calculated using the ROC analysis.
All analyses were performed by using SAS statistical software (V.9.4; SAS Institute) for data cleaning and imputation, and R V.4.2.2 for model parameters estimation and graphing (riskRegression, ggDCA and regplot packages). A two-tailed p<0.05 was considered statistically significant.
Results
Study cohort and incidence rates
Study cohort and incidence rates of the 8372 participants screened in the ACURE dataset, 4065 patients were eligible and finally included in analyses. The study flow is illustrated in figure 1. A comparable number of participants were assigned to the development cohort (n=2196) and validation cohort (n=1869).
The characteristics of the participants in development and validation cohorts are illustrated in table 1. Notable differences in demographic and clinical features were observed between the two groups. The cumulative incidence of outcomes between the two groups was without significant difference. The 12-month severe exacerbation incidence rates were 11.55% (95% CI 10.06% to 13.16%) and 12.30% (95% CI 10.67% to 14.05%), respectively. The COPD-specific readmission incidence rates were 11.31% (95% CI 9.83% to 12.91%) and 12.26% (95% CI 10.63% to 14.02%), respectively.
Competing risks regression
The multivariable coefficient estimates for predictors are presented in table 2. Fourteen predictors were selected by selection criteria. The nomograms built based on the prediction models to predict the 1-month, 6-month and 12-month probabilities of severe exacerbations and COPD-specific readmissions are shown in figure 2.
Model performance evaluation and decision curve analysis
The C-indexes of SERCO for severe exacerbations at 1 month, 6 months and 12 months were 77.3 (95% CI 70.7 to 83.9), 76.5 (95% CI 72.6 to 80.4) and 74.7 (95% CI 71.2 to 78.2), respectively (table 3). Similar C-indexes were achieved at the internal validation set. In the external validation set, the C-indexes were relatively lower at 60–70 levels. Similar C-indexes for predicting COPD-specific readmissions were achieved. Compared with the history of severe exacerbations in the past 12 months, SERCO has better discrimination ability (online supplemental figure S1). Predictor coefficients in the original prediction model were adjusted using slopes and intercepts driven from the calibration plots.22 Table 4 demonstrates the slopes and intercepts before and after adjustment. Online supplemental figure S2 and figure 3 illustrate the agreement of predicted and the observed probabilities. In the external set, SERCO underestimates the probabilities in lower-risk groups while overestimating in the higher-risk groups (figure 3).
The decision curve analysis indicates that the SERCO models have net benefits within the predictive probability threshold range of 1.5%–5% for 1-month outcomes, 5%–25% for 6-month outcomes and 7%–35% for 12-month outcomes (figure 4). About 35–70 out of 100 patients can thus be detected by applying SERCO model.
The cut-off values, sensitivity, specificity, positive and negative predictive values of SERCO are shown in online supplemental table S5. According to the cut-off value of 12-month outcomes, participants were stratified into low-risk and high-risk groups, with the cut-off predicted 12-month risk at 16%. The median cumulative incidence of severe exacerbations and COPD-specific readmissions for low-risk patients versus high-risk patients were significantly differentiated (online supplemental figure S3).
Clinical application
For clinical application, an online platform was developed so the clinicians or patients can evaluate the predicted risk of AECOPD by visiting the website and inputting the individual characteristics: https://www.qunxiangjiankang.tech/copd.
Discussion
The most important finding of the study was the development and validation of SERCO to predict short-term, medium-term and long-term risk of severe exacerbations and COPD-specific readmissions within 12 months after discharge in patients hospitalised for AECOPD. SERCO used predictors routinely available during hospitalisation, and performances were superior to prior severe exacerbation history,6 7 which will generate more accurate individualised predictions that allow clinicians to risk-stratify patients. To increase clinical practicability, SERCO provides concurrently evaluation of severe exacerbation and COPD-specific readmission risk in patients hospitalised for AECOPD. To our knowledge, SERCO is the first model to focus on hospitalised treatment in predicting exacerbation risk of AECOPD. When the availability of predictors and practical applicability were considered, SERCO showed promise for clinical implementation but need more validation among more extensive populations in real-world settings in China. Whether SERCO models in this study are applicable only to China or also to other countries needs to be verified in the future.
Among existing prognostic prediction models, the study population, investigation settings, involved predictors and model performance measures were inconsistent.24–31 Most prognostic models were developed in outpatients with COPD and the most prevalent endpoint was mortality, followed by exacerbation risk.6 32 The study population of two exacerbation prediction models were outpatients with COPD rather than inpatients.24 26 In our study, SERCO provides individual severe exacerbation risk prediction based on the population admitted for AECOPD.
We are aware of the scant of COPD-specific risk-prediction tools to identify and stratify the readmission risk in inpatients with AECOPD. Two previous studies focused on predictive models for the risk of readmission due to COPD exacerbations, but both were based on retrospective data and lacked external validation and risk stratification.31 33 The CODEX index was a useful predictor for all-cause rather than COPD-specific risk of survival and readmission both at 3 months and 1 year after discharge.25 The PEARL score was a model to predict and stratify patients’ risk of 90-day readmission/death in patients hospitalised with AECOPD.29 No readmission prediction tools focused on treatment during hospitalisation, which not only correlates with the severity of the patient’s condition but may also influence the patient’s treatment pattern after discharge. SERCO first considers the hospitalised treatment and enables us to assess the short-term, medium-term and long-term risk of readmission for AECOPD among patients hospitalised for AECOPD.
The reported predictors were divergent in prediction models.11 A portion of the predictors in this study is consistent with those previous studies reported such as BMI, smoking status, FEV1%pred, previous admissions and presence of comorbidities.11 24 34 35 The novel predictors included methylxanthine treatment and non-use of inhaled short-acting bronchodilators during hospitalisation and consistent usage of long-acting bronchodilators exceeding 3 months. Methylxanthine is a moderate bronchodilator, which can improve FEV1 and relieve breathlessness when combined with inhaled long-acting bronchodilators.36 Intravenous methylxanthines are not recommended in patients with AECOPD due to the side effects.7 However, inhaled drugs may not be available and affordable in some regions of China, and methylxanthine may be an affordable add-on therapy.36 This finding may be related to the healthcare system in China and the practice of initiating treatments on hospitalisation. Another important aspect of our study is that we include the interactions in the prediction model. Although the underlying mechanisms of the interactions on COPD exacerbation are unclear, but the interactive effects contribute to the predictive capabilities and should be noted in risk evaluation. We also tried to assess the non-linear relationships between continuous variables such as age, BMI and FEV1%pred with outcomes. FEV1%pred was a predictor with a non-linear association with outcomes while its transformed polynomial variable had weak prediction capacity in the model. The SERCO model showed good discrimination and calibration in the internal validation cohort although it overestimated outcome probabilities in their mid-range but only by approximately 2%. The model showed weak performance in the external validation cohort. The unsatisfactory performance might be due to the disparities in the distributions of the predictors. The two cohorts were from different geographical regions in China, which means the demographic profiles and the treatment of COPD might be very different. It is noted that in the external validation cohort, a higher proportion of participants had previously been diagnosed with COPD and more frequently exacerbated in the past 12 months. The treatment during hospitalisation also differed in the two cohorts. It is supposed that the model may be adapted to external clinical settings, but the performance would be affected by patients’ profiles. We adjusted the coefficients in the original model using slope and intercept derived from the developing cohort. The adjustment has a positive effect, although it is poor, on the external calibration (see online supplemental table S6). Poor calibration in the external dataset may make a prediction model clinically useless or even harmful.22 Further models using more optimal approaches are needed to be developed to address the poor external calibration.
This study has several limitations. First, in this study, a single statistical approach was applied for modelling, limiting the comparison of model performance among multiple approaches such as machine learning. We acknowledge that the modelling strategy we used may not be the optimal method. Further modelling exploration for better performance is required in the future. Second, to maximally eliminate the bias of recall, we only asked questions and recorded the history of severe exacerbations in our study, lacking data on moderate COPD exacerbations ((prescribe antibiotics or corticosteroids (or both)).7 Some recognised predictors, such as eosinophil count, were not included in the model. A notable proportion of patients had missing eosinophil count values in the dataset (35.18%, 1430/4065), and there would be a risk of bias if all data with missing values were deleted. We also acknowledge that a notable number of patients (1140) were excluded due to a lack of follow-up. We did not perform an external validation by an independent cohort. However, using data from different regions was an alternative way, and there were notable differences between the development and validation cohorts in the study. Model updating and re-examination of its external validity will be necessary when new data sources become available. Additionally, it will overestimate the severity of airway limitation if spirometry was done in the exacerbation state, especially in never-diagnosed patients. Therefore, to reduce this bias, all hospitalised patients with AECOPD underwent spirometry before their condition improved before discharge. In addition, the results of previous pulmonary function tests 6 months before hospitalisation and 30 days to 6 months after discharge were used as supplements.
Conclusions
For the population cohort on which the model was internally validated, SERCO enables clinicians to predict individual short-term, medium-term and long-term risks of severe exacerbations and COPD-specific readmissions in patients hospitalised for AECOPD. We stratified the risk for the convenience of clinical application. SERCO has the potential to be used as a decision tool for personalised treatments and hospital discharge plans. Further model development and validation are needed across a wide range of clinical settings in China and other countries to confirm that SERCO is suitable for routine clinical use.
Data availability statement
Data are available on reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the ethics committee of the China-Japan Friendship Hospital (No. 2015-88). Informed consent was obtained from all participants. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
This work was previously presented as an abstract at the 27th Congress of the Asian Pacific Society of Respirology, 16 November 2023–19 November 2023, Singapore:
Respirology 2023;28:3–425. https://onlinelibrary.wiley.com/doi/10.1111/resp.14618,
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors TY and CW designed the study and had full access to all the data in the study. TY took responsibility for the integrity of the data and the accuracy of the data analysis. YW and RH contributed substantially to the study design, data analysis and interpretation, and manuscript writing. XR, KH, JL, HN, WL, FD and BL contributed substantially to the study design, data collection and interpretation. TY and CW are the guarantors of this study.
Funding This work was supported by the CAMS Innovation Fund for Medical Sciences (CIFMS) (grant number: 2021-I2M-1-049).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.