Critical Care•

Dynamic nomogram for predicting acute kidney injury in patients with community-acquired pneumonia

Abstract

Introduction Acute kidney injury (AKI) is a common complication in patients with community-acquired pneumonia (CAP) and negatively affects both short-term and long-term prognosis in patients with CAP. However, no study has been conducted on developing a clinical tool for predicting AKI in CAP patients. Therefore, this study aimed to develop a predictive tool based on a dynamic nomogram for AKI in CAP patients.

Methods This retrospective study was conducted from January 2014 to May 2017, and data from adult inpatients with CAP at Nanjing First Hospital were analysed. Demographic data and clinical data were obtained. The least absolute shrinkage and selection operator (LASSO) regression model was used to select important variables, which were entered into logistic regression to construct the predictive model for AKI. A dynamic nomogram was based on the results of the logistic regression model. Calibration and discrimination were used to assess the performance of the dynamic nomogram. A decision curve analysis was used to assess clinical efficacy.

Results A total of 2883 CAP patients were enrolled in this study. The median age was 76 years (IQR 63–84), and 61.3% were male. AKI developed in 827 (28.7%) patients. The LASSO regression analysis selected five important factors for AKI (albumin, acute respiratory failure, CURB-65 score, Cystatin C and white cell count), which were then entered into the logistic regression to construct the predictive model for AKI in CAP patients. The dynamic nomogram model showed good discrimination with an area under the receiver operating characteristics curve of 0.870 and good calibration with a Brier score of 0.129 and a calibration plot. The decision curve analysis showed that the dynamic nomogram prediction model had good clinical decision-making.

Conclusion This easy-to-use dynamic nomogram may help physicians predict AKI in patients with CAP.

What is already known on this topic

  • Acute kidney injury (AKI) is common and negatively affects both short-term and long-term prognosis in patients with community-acquired pneumonia (CAP). However, there is no study on developing an easy-to-use clinical tool for predicting AKI in patients with CAP.

What this study adds

  • The dynamic nomogram model, including five important predictors (albumin, acute respiratory failure, CURB-65 score, Cystatin C and white cell count), shows good discrimination and calibration performance for predicting AKI in patients with CAP.

How this study might affect research, practice or policy

  • This clinical tool may benefit patients at high risk of AKI through early detection and timely intervention.

Introduction

Community-acquired pneumonia (CAP) is a leading cause of morbidity and mortality, as well as a significant economic burden on individuals and society.1 2 Acute kidney injury (AKI) is a common complication in patients with CAP, with incidence rates ranging from 18% to 34%.3–6 Even in patients with non-severe CAP, the incidence of AKI is nearly as high as 16%–25%.7 Moreover, AKI negatively affects both the short-term and long-term prognosis of CAP patients. CAP patients who developed AKI were likely to have an increased risk of in-hospital and 30-day mortality, require intensive care unit admission, non-invasive and invasive mechanical ventilation, and have a longer hospital stay than those without AKI.4 5 Patients with CAP and AKI have worse postdischarge outcomes (permanent loss of renal function, dialysis and death) than patients with CAP alone.8

AKI is a common and severe clinical syndrome with adverse outcomes, but no effective treatment methods have been developed.9 Early detection and timely intervention are still effective ways to improve outcomes for patients with AKI.10 However, there is no research on developing a clinical tool for predicting AKI in CAP patients. Therefore, this study aims to establish a predictive tool based on a dynamic nomogram for AKI in CAP patients.

Methods

Study population and design

This retrospective study was conducted at Nanjing First Hospital (Nanjing, China) from January 2014 to May 2017. All patients (age ≥18 years) discharged with a primary diagnosis of CAP were included in this study. The exclusion criteria were as follows: (1) history of end-stage renal disease or regular dialysis; (2) less than two repeated serum creatinine (SCr) measurements and (3) incomplete medical records. Patient consent was waived due to the retrospective nature of the study design. We confirmed that the data were anonymised and maintained confidentiality, and the study was conducted in accordance with the Declaration of Helsinki.

Definitions of cap and AKI

The criteria for diagnosing pneumonia based on the detection of interstitial infiltrate changes on chest CT or radiography in patients with at least one of the following conditions: (1) recent presence of sputum, cough or dyspnoea, (2) peripheral white cell counts >10×109/L or <4×109/L or (3) temperature >38.0°C.11 In addition, illness onset was specifically in the community rather than in the healthcare setting.

AKI was diagnosed based on the Kidney Disease Improving Global Outcomes (KDIGO) criteria, which defined AKI as an increase in SCr levels by ≥1.5 times from baseline within 7 days of illness onset or an increase in SCr levels by ≥0.3 mg/dL (26.4 µmol/L) within 48 hours of illness onset.12 The lowest value of SCr measured during hospitalisation was defined as the baseline SCr value. The urine output standard could not be considered due to a lack of complete urine output data.

Data collection

All data in this study were collected from the medical records: demographics (gender and age), comorbid conditions (hypertension, diabetes mellitus, coronary artery disease, cardiac insufficiency, atrial fibrillation, chronic obstructive pulmonary disease (COPD), chronic kidney disease, chronic cor pulmonale, tumour and cerebrovascular diseases), complications (acute respiratory failure and AKI), severity scoring (confusion, urea nitrogen, respiratory rate, blood pressure and age 65 years or older (CURB-65) score),13 and laboratory tests (Cystatin C, albumin, total protein, prealbumin, haemoglobin, platelet count and white cell count). If the laboratory tests were measured multiple times during hospitalisation, we selected the first test value within 48 hours after admission.

Feature selection

We initially selected the important variables associated with AKI in CAP patients to construct the prediction model. To minimise potential overfitting brought by the high dimensionality of the features, all potentially important variables (p<0.05) were selected by univariable analysis and then entered into the least absolute shrinkage and selection operator (LASSO) regression with R package glmnet (V.4.12). LASSO regression was performed to identify important variables. LASSO selected variables by shrinking the coefficients of less-important variables from logistic regression to zero.14

Model development and visualisation

To develop an AKI prediction model in CAP patients, all data were randomly divided into a training set (70%) and a testing set (30%). The training set was used to develop the model, while the testing set was used to evaluate its performance. The retained variables with non-zero coefficients in LASSO regression were included in the logistic regression, and the model was then developed to determine the association between independent risk factors and AKI. Moreover, SHapley Additive exPlanations (SHAP) was used to interpret feature importance in the model with logistic regression. Finally, to achieve logistic regression model visualisation, we established a dynamic nomogram based on the result of the logistic regression model. Shiny was applied to build an interactive web-based dynamic nomogram application (V.1.7.2).

Model evaluation

The fivefold cross-validation was performed for the logistic regression model derivation and internal evaluation by dividing the training set into five mutually exclusive parts, four of which were used as training data for the model derivation and one for evaluation as inner validation data. This process was repeated five times to generate five different but overlapping training data sets and five unique validation data sets. The predictive performance of the model was evaluated using the receiver operating characteristics (ROC) curve, calibration curve, decision curve, learning curve, and evaluation indicators, including area under the ROC curve (AUC), sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV) and F1-score. Model discrimination was evaluated using AUC.15 Model calibration was assessed using the calibration plot and Brier score, which ranges from 0 to 1. The lower Brier score indicated that the better model prediction was calibrated.16 The decision curve was applied to determine the model’s clinical applicability.17 A learning curve was employed to check whether the model was overfitting.18

Statistical analyses

All statistical analyses were performed using SPSS V.22.0 (IBM), R V.3.6.3 and Python V.3.7. Continuous variables were expressed as the median (IQR) for non-normally distributed data. Continuous variables were presented as the mean and SD for normally distributed data. Categorical variables were expressed as frequencies and percentages. The χ2 or Fisher’s exact test was performed for categorical variables. The differences in continuous variables (non-normally distributed data) were assessed by the Mann-Whitney U test, and the differences in continuous variables (normally distributed data) were evaluated using the Student’s t-test. The logistic regression model was performed with the open-source Python package scikit-learn V.0.22.1. SHAP was implemented using the shap python package V.0.39.0.

Results

Patient characteristics

Totally, 2883 patients with CAP were enrolled in this study (online supplemental figure S1). The median age was 76 (IQR, 63–84) years, and 61.3% were male. Of all patients with CAP, 827 (28.7%) developed AKI. Table 1 demonstrates that the characteristics of the patients between the training (n=2018, 70%) and testing (n=865, 30%) sets were well balanced.

Table 1
|
Characteristics data: training set versus testing set

Feature selection

A total of 21 variables were included in this study. After univariable analysis for comparison between the non-AKI group and the AKI group, there were 18 variables (p<0.05) remaining that further underwent feature selection by the LASSO regression (table 2). LASSO regression showed that the five features (albumin, acute respiratory failure, CURB-65, Cystatin C and white cell count) were associated with AKI in patients with CAP (figure 1).

Figure 1
Request permissionrequest permission icon
Figure 1

Selection of predictors using the LASSO regression analyses. LASSO coefficient profiles of the features (A) and feature selection for the predictive model (B). The vertical axis shows the model misclassification rate, and the horizontal axis shows log (λ). Two vertical lines are drawn at the selected value using cross-validation, and optimal values are obtained by applying the minimum criteria and 1 SE of the minimum criteria (the 1-SE criteria). For the cross-validation results, we selected the 1-SE criteria to determine the final five candidate variables. LASSO, least absolute shrinkage and selection operator.

Table 2
|
Univariate logistic regression for AKI in the training set

Model development and visualisation

After multivariate logistic regression, we found that Cystatin C (OR 5.42, 95% CI 4.23 to 7.02; p<0.001), CURB-65=1 (OR 1.90, 95% CI 1.15 to 3.28; p=0.016), CURB-65=2 (OR 3.22, 95% CI 1.93 to 5.60; p<0.001), CURB-65≥3 (OR 4.41, 95% CI 2.48 to 8.11; p<0.001), albumin (OR 0.93, 95% CI 0.91 to 0.96; p<0.001), acute respiratory failure (OR 3.40, 95% CI 2.51 to 4.60; p<0.001) and white cell count (OR 1.07, 95% CI 1.04 to 1.10; p<0.001) were still significantly associated with AKI in patients with CAP (table 3). The relationship between the value of each feature (red was high, blue was low) and the predicted risk of AKI by SHAP was exhibited in figure 2A. The importance of the features was ranked from high to low: Cystatin C, CURB-65, albumin, acute respiratory failure and white cell count. Cystatin C was the most important, and white cell count was the least important. Then, the logistic model based on the five independent predictors was developed as a nomogram. Each predictor was assigned a score. By calculating the total score from all the predictors and locating it on the total point scale, we could obtain the probabilities of the predicted AKI risk by drawing a vertical line to the total score (figure 2B). However, the nomogram was not easily applied in clinical practise because of the imprecision of the corresponding points for each predictor and the probability associated with the total points. Therefore, we developed a clinician-friendly and online dynamic nomogram. By entering the relevant values of predictors in the interactive interface and clicking the ‘predict’ button, the dynamic nomogram displayed a graphical representation of the probability of the predicted AKI risk. Additionally, by clicking on the ‘numerical summary’ button, it showed specific probabilities and 95% CIs (figure 2C).

Figure 2
Request permissionrequest permission icon
Figure 2

SHAP plot, nomogram and dynamic nomogram. The SHAP plot demonstrates the importance of each factor for AKI in the logistic regression model (A). The colour bar on the right indicates the relative value of a factor in each case. Red represents higher values, and blue represents lower values. Establish a nomogram for the prediction of AKI in patients with CAP (B). The nomogram was developed in the training set by incorporating the following five parameters: Cystatin C (mg/L), CURB-65 (0: CURB-65 score=0, 1: CURB-65 score=1, 2: CURB-65 score=2, 3: CURB-65 score≥3), albumin (g/L), acute respiratory failure (0: without acute respiratory failure, 1: with acute respiratory failure) and white cell count (WCC) (109/L). Online dynamic nomogram accessible at https://cap-aki.shinyapps.io/cap-aki/, depicting an example for predicting the probability of AKI in patients with CAP (C). AKI, acute kidney injury; CAP, community-acquired pneumonia; CURB-65, confusion, urea nitrogen, respiratory rate, blood pressure and age 65 years or older; SHAP, SHapley Additive exPlanations.

Table 3
|
Multivariate logistic regression for AKI in the training set

Model evaluation

The AUC, sensitivity, specificity, accuracy, PPV, NPV and F1-score of the model in the training set, validation set and testing set were shown in table 4. To obtain a robust estimate of expected performance with unseen data, we applied five k-fold cross-validation to calculate the AUCs of the training (figure 3A) and validation datasets (figure 3B). The model learning curve showed that the training AUC gradually came close to the test AUC when the number of training examples increased (figure 3C), indicating that the model discrimination performance grew steadily with more training examples provided, and the model did not suffer from major overfitting. Furthermore, compared with the AUCs of the training (0.878) and validation (0.877) datasets, the value of the AUC in the testing set (0.870) (figure 3D) was similar, indicating that the model was robust. The model calibration performance assessed using the Brier score (0.129) and calibration plot (figure 3E) visually demonstrated good calibration in the testing set. Decision curve analysis was shown in figure 3F, and the decision curve using the dynamic nomogram prediction model had good clinical decision-making.

Figure 3
Request permissionrequest permission icon
Figure 3

Evaluation of the logistic regression model. Fivefold cross-validation receiver operating characteristic (ROC) curve in the training set (A); fivefold cross-validation ROC curve in the validation set (B); model learning curve (C); ROC curve in the testing set (D); calibration plot in the testing set (E); decision curve in the testing set (F).

Table 4
|
Model evaluation metrics in training, validation and testing sets

Discussion

In this study, a dynamic nomogram is developed to predict the individual risk of AKI in CAP patients. To the best of our knowledge, this is the first study to develop a convenient and practical dynamic nomogram for predicting AKI in CAP patients. This dynamic nomogram includes albumin, acute respiratory failure, CURB-65, Cystatin C and white cell count, making it an objective, visual and simple-to-use screening tool for AKI in CAP patients. A dynamic nomogram based on a user-friendly digital interface responding in a dynamic online manner to personalised medicine may help support better clinical decision-making. Furthermore, the dynamic nomogram shows good discriminatory ability, calibration performance and clinical efficacy for predicting AKI in CAP patients.

The incidence of AKI was 28.7%, similar to the incidence rates reported by other studies.3 4 6 7 Murugan et al reported that 34.4% of patients developed AKI in the multicentre prospective cohort study of 1836 patients hospitalised with CAP.3 Akram et al reported that the incidence rate of AKI on admission in CAP patients was 18%.4 Latief et al observed in a prospective observational study that 27.6% of CAP patients had AKI.6 A previous study reported that the incidence of AKI was as high as 16%–25% in patients with non-severe pneumonia.7 As AKI has a poor impact on both short-term and long-term prognosis in CAP patients, several studies have investigated the risk factors for AKI in CAP patients.4 5 8 Independent factors associated with AKI reported in these patients include age, male gender, comorbidity (such as chronic kidney disease, hypertension, diabetes and cardiac dysfunction), blood parameters (C reactive protein, interleukin-6, tumour necrosis factor and lactate dehydrogenase), acute respiratory failure, drugs (such as statins, ACE inhibitors, angiotensin-II-receptor blockers, diuretics and vasoactive drugs), and severity scoring systems of pneumonia (Pneumonia Severity Index19 and CURB-65).3–7 Albumin, Cystatin C, white cell count, CURB-65 and acute respiratory failure were identified as five independent factors associated with AKI in CAP patients in our study. Acute respiratory failure and CURB-65 have been identified as predictors for AKI in CAP patients in previous studies.5 6 Although white cell count, albumin and Cystatin C have not been reported as independent factors for AKI in CAP patients, they have been reported as predictors for AKI in other clinical settings.20–25 We discovered that Cystatin C is the most important predictor for AKI in CAP patients, and a one-unit (1 mg/L) increment in Cystatin C is associated with a 5.42-fold increased risk of AKI.

Although several studies have investigated predictors for the early detection of AKI in CAP patients, few have been applied in clinical practise. This is the first study to use a convenient and practical dynamic nomogram to predict AKI in CAP patients. To apply the dynamic nomogram in clinical practise, we developed a clinician-friendly and online dynamic nomogram (https://cap-aki.shinyapps.io/cap-aki/). By entering the relevant values of predictors in the interactive interface and clicking the ‘Predict’ button, a dynamic nomogram displayed a graphical representation of the probability of the predicted AKI risk. Additionally, by clicking on the ‘Numerical summary’ button, it showed specific probabilities and 95% confidence intervals. The dynamic nomogram model shows good discrimination with an AUC of 0.870 (sensitivity=85.3% and specificity=76.4%) and good calibration with a Brier score of 0.129. Moreover, decision curve analysis reveals that the dynamic nomogram prediction model is clinically useful. The dynamic nomogram was more intuitive and convenient in its actual application due to the result’s visualisation. However, the predicted results of the dynamic nomogram were for reference only and should not be used as the sole basis for decision-making. Doctors also needed to make decisions based on clinical experience and other professional knowledge.

However, this study is not devoid of limitations. First, the dynamic nomogram for AKI in patients with CAP was fit in a single-centre retrospective cohort, and the generalisability and predictive accuracy need to be validated in a prospective multicentre cohort. Second, according to KDIGO criteria, the diagnosis of AKI is typically based on the SCr and urine output levels.12 However, due to the unavailability of urine output data, the definition of AKI according to the urine output standard is not included in our analysis. Third, some novel biomarkers (such as proenkephalin, Dickkopf-3, and C-C motif chemokine ligand 14) have been recently identified and applied to predict AKI.26–30 If these novel biomarkers are combined, the predictive performance of the model may be further enhanced. Fourth, AKI is commonly encountered in patients with decompensated cirrhosis. As this is a retrospective study, we lack data on liver disease in this model.31 Fifth, although we find that Cystatin C is an important biomarker for predicting AKI in the model, Cystatin C may not be widely used in some areas, which may limit the application of the dynamic nomogram.

In conclusion, our study uses a dynamic nomogram established from five independent predictive factors (Cystatin C, CURB-65, albumin, acute respiratory failure and white cell count) to predict the risk of AKI in CAP patients. Patients at high risk of AKI may benefit from this clinical tool through early detection and timely intervention.

Overview

Article metrics
Altmetric data not available for this article.
Dimensionsopen-url