Introduction Chronic obstructive pulmonary disease (COPD) is a chronic respiratory disease that may be punctuated by episodes of worsening symptoms, called exacerbations. Acute exacerbations of COPD (AECOPD) are detrimental to clinical outcomes, reduce patient quality of life and often result in hospitalisation and cost for the health system. Improved diagnosis and management of COPD may reduce the incidence of hospitalisation and death among this population. This scoping review aims to identify improvement interventions designed to standardise the hospital care of patients with AECOPD at presentation, admission and discharge, and/or aim to reduce unnecessary admissions/readmissions.
Methods The review followed a published protocol based on methodology set out by Arksey and O’Malley and Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Electronic database searches for peer-reviewed primary evidence were conducted in Web of Science, EMBASE (Elsevier) and PubMed. Abstract, full-text screening and data extraction were completed independently by a panel of expert reviewers. Data on type of intervention, implementation supports and clinical outcomes were extracted. Findings were grouped by theme and are presented descriptively.
Results 21 articles met the inclusion criteria. Eight implemented a clinical intervention bundle at admission and/or discharge; six used a multidisciplinary care pathway; five used coordinated case management and two ran a health coaching intervention with patients.
Conclusion The findings indicate that when executed reliably, improvement initiatives are associated with positive outcomes, such as reduction in length of stay, readmissions or use of health resources. Most of the studies reported an improvement in staff compliance with the initiatives and in the patient’s understanding of their disease. Implementation supports varied and included quality improvement methodology, multidisciplinary team engagement, staff education and development of written or in-person delivery of patient information. Consideration of the implementation strategy and methods of support will be necessary to enhance the likelihood of success in any future intervention.
- COPD exacerbations
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What evidence is there for initiatives which aim to improve or standardise the acute care of patients with acute exacerbation of chronic obstructive pulmonary disease (AECOPD) at presentation, admission or discharge, and/or aim to reduce unnecessary AECOPD admissions/readmissions?
This scoping review consolidates the published evidence regarding hospital-based interventions which aim to standardise care at presentation, admission and/or discharge or to reduce unnecessary admission or readmissions for patients with AECOPD.
This review provides a narrative synthesis of the evidence for front-line service providers, payers and planners designing improvement initiatives for AECOPD care and facilitates discussion of the implementation strategy and methods of support that will be necessary.
Chronic obstructive pulmonary disease (COPD) is a common, preventable and treatable respiratory disease characterised by persistent respiratory symptoms.1 Although misdiagnosis and underdiagnosis is common,2–4 it is estimated that worldwide prevalence of COPD has increased by 44.2% between 1990 and 2015, when the global prevalence was estimated to be 174.5 million individuals.5 COPD is currently the third-leading cause of death globally.6 7
The clinical course of COPD is punctuated by episodes of acute worsening of symptoms. These acute exacerbations of COPD (AECOPD) often require hospitalisation and are costly in terms of economics,8–10 lung health11 and quality of life (QoL) for the patient12 with heightened risk of readmission noted among patients with certain comorbidities and conditions, such as heart failure, renal failure, depression and alcohol use.13 Mortality rates of patients at 12 months postdischarge due to hospitalisation for AECOPD are over 20%.14 15 Huge variations in care have been noted within and between European countries with low adherence to clinical management recommendations.16 The need to reduce COPD exacerbations and hospitalisations has been recognised by guideline development bodies, researchers and clinicians,17 with international consensus guidelines calling for implementation of evidence-based approaches for improved COPD diagnosis and management.18 WHO ‘25 by 25 goal’ aims to reduce global deaths from COPD by 25% by 2025.19
The purpose of this review is to identify initiatives which aim to improve standardise hospital-based care of AECOPD patients at presentation, admission or discharge, and/or aim to reduce unnecessary AECOPD admissions/readmissions. These findings will assist in the design of a national AECOPD initiative which has been commissioned to standardise AECOPD acute, hospital-based care across Ireland.
Scoping reviews are a type of knowledge synthesis which present a broad overview of the available evidence, irrespective of study quality.20 Scoping reviews are useful to clarify key concepts and identify gaps when examining emerging areas,21 and as such was deemed an appropriate methodology for this review. The protocol for this scoping review, based on the methodological framework proposed by Arksey and O’Malley22 and Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines,23 has previously been published.24
Identifying relevant studies
A search was undertaken for COPD studies relating to acute hospital-based care presentation, admission and discharge interventions as well as interventions aiming to reduce COPD admission(s) or readmission(s). English language, peer-reviewed studies published between January 2000 and September 2020 in the Web of Science, Embase (Elsevier) and PubMed electronic databases were included. A rapid scoping review undertaken in 2017 by this research team detected no relevant results between 1990 and 2000. Examples of specific search terms used include ‘COPD, intervention, quality improvement (QI), acute care, admission avoidance, prevention of readmission, admission and discharge bundles, care pathways’. A full list of search terms is contained in online supplemental materials. The participants/population, intervention(s), context and outcomes (PICO) for this review are presented in table 1 below.
Relevant articles were screened as previously described.24 RM, LP and OW conducted independent title reviews against PICO criteria. All authors participated in abstract and full-text review. Any inclusion/exclusion discrepancies were resolved by discussion and consensus between RM, OW and SW. Eligibility criteria are listed in table 2. Studies were included where endpoints were described.
Two researchers (RM and LP) designed a standardised Microsoft Excel datasheet for data extraction which was validated (RM and OW) using two randomly selected articles from the search results. Data items were extracted from each paper using the headings described previously24; study descriptors, study design, intervention descriptors, measures, results, discussion and reviewer’s appraisal.
Patient and public involvement
It was not appropriate or possible to involve patients or the public in the design, or conduct, or reporting, or dissemination plans of our research.
Of the total 1922 records, after removal of duplicates (n=324), 1466 were removed following abstract screening, with a further 111 removed at full-text screening, resulting in 21 articles included for analysis (figure 1).
Implementation strategies reported in included studies focused on training and education of healthcare professionals (n=11) and patients with AECOPD (n=15). Some studies engaged multidisciplinary teams (MDT) to support change implementation (n=13) or used named champions/coordinators to disseminate the change/change message (n=11).
Most of the studies reported positive trends, with some showing significant change and others non-significant, in primary outcome such as intervention compliance (n=11), reduced length of stay (LOS) (n=4) or reduced readmissions (n=7). Other improvements included reduced mortality (n=3) and improved QoL for patients (n=4). Table 3 summarises the included paper characteristics.
A variety of methods, implementation strategies and supports were used to improve AECOPD care, demonstrating varying levels of success. Content analysis of the final 21 articles identified four intervention types that were used to impact care at presentation/admission or discharge, or both, for patients with AECOPD. These are (1) clinical care bundles (n=8; defined as groups of improvement interventions which are implemented together25), (2) care pathways (n=6; defined as written or computer-based systems which support clinical decision making and the organisation of care processes for patients26), (3) coordinated case management (n=5 defined as a process in which a designated person supports the coordination, integration and management of a patient’s health and social care needs27) and (4) health coaching (n=2; defined as a patient-centred partnership between patient and trained coach where patients are encouraged to determine their personal health goals and increase knowledge and confidence in their own ability to manage their condition28). Methodologies applied are presented in table 4.
Implementation of, or improved compliance with, AECOPD care bundles was the focus of eight studies. Of these, two were aimed at presentation or admission,29 30 four introduced a discharge bundle,31–34 one implemented both an admission and discharge bundle35 and one described an end-to-end bundle covering care from presentation through to discharge.36
Of the three articles concerned with a presentation or admission bundle, one used an MDT designed 10-step bundle29 and the other two used existing BTS guidelines.30 35 McCarthy et al found that staff education improved compliance significantly from a mean of 4.6–7 elements completed but without significant reduction in the 30-day readmission rate or median LOS.29 Two studies employed QI methods30 35 with one reporting increased adherence from 63% to 77% in 2 months, remaining above 70% for the next 4 months.30 Success was attributed to multiple communication strategies to raise the profile of the bundle, such as posters, emails and engagement meetings. In the other paper, training, networking and mentoring resulted in staff rating the use of bundles positively, although no improvement in readmission or emergency department (ED) presentation rates occurred.35
Hopkinson et al detailed the development and implementation of a COPD discharge care bundle,31 which was later spread to nine acute hospitals across England.32 These studies used QI tools and methodologies such as process mapping, stakeholder engagement and rapid-cycle plan-do-study-act (PDSA) testing. Both also engaged the MDT (ward nurses, physiotherapists, clinical nurse specialists and doctors) in activities including education meetings, information stands, daily pharmacist teaching, aide-mémoire development, weekly check-ins with staff and performance-related prizes to improve bundle awareness and compliance. The initial hospital study saw increased compliance with regard to referrals to smoking cessation services and pulmonary rehabilitation sessions, self-management plan provision and medication review.31 Thirty-day readmissions saw a non-significant reduction from 16.4% to 10.8%. In the follow-on study, results showed a similar non-significant reduction in the 28-day readmission rate; while within the readmission group, a further, non-significant reduced LOS of 2 days was noted in the intervention group.32
Using QI methods including multiple PDSA tests with staff and patient feedback, and a redesigned patient pathway to standardise care processes, improve discharge planning and give healthcare professionals greater role clarity, Zafar et al noted that bundle compliance increased to 90%.33
Epstein et al integrated a clinical decision support tool with an existing electronic healthcare record to improve clinician adherence to AECOPD discharge recommendations.34 As a result, more patients were discharged with the correct recommendations (80.47% vs 25.37%). Patients were far more likely to receive prescribed medication within 60 days of discharge (54% vs 20%) and demonstrated increased vaccine uptake (92% vs 13%), while follow-up visits were provided to nearly 98% of patients.
Pendharkar et al held engagement meetings and initiated a new Computerised Physician Order Entry (CPOE) for AECOPD in a large, tertiary care teaching hospital.36 The bundle included elements for tests, medications, consultations and discharge planning with key elements prechecked and was implemented with different hospital physician groups (hospitalist, general internist or respiratory specialist) admitting AECOPD patients. Though the voluntary CPOE was used by the physicians less frequently than anticipated, when it was used LOS was reduced by 1.15–1.8 days. Importantly, readmission rates did not increase, indicating that earlier discharge did not have a negative impact on the safety of patients at home.
Six studies introduced or evaluated an AECOPD care pathway. All interventions were MDT-designed and implemented; with three employing a designated coordinator.37–39 All included criteria for investigations, treatment interventions, consultations with multiple support disciplines (eg, physiotherapy) and discharge planning. Patient education was an additional priority in four studies.38 40–42 Implementation supports included time and discipline specific prompts,40 a scoring system to aid in decision making around admission need,41 a printed flowsheet to identify sequential treatment steps42 or coordinated clinical audit, workshops, teaching sessions and meetings with pathway facilitators.38 LOS was frequently measured and results varied from no change38 to a non-significant 0.89-day improvement40 to significant reductions of 2 days37 and 4 days.41 Impact on readmission rate, if recorded as a primary outcome, varied between studies. McManus et al noted that their score-based admission decision model was associated with a 4% drop in 1-month readmission rate and a 57% reduction in-hospital mortality.41 Vanhaecht et al recorded a significantly reduced readmission rate of 27.3% (down from 33%) at 30 days.38 In the study conducted by Ban et al, a longer time between admissions was observed, although no significant reduction in readmission occurred.37 Additional improvements were recorded in several studies, including adherence to key clinical interventions,38 39 41 42 improved confidence of ward staff after education regarding inhaler technique, smoking cessation and pulmonary rehabilitation,31 improved teamwork or communication40 41 and a positive impact on patient perception of their ability to self-manage their disease.40
Coordinated case management
Five studies implemented coordinated care for AECOPD discharge and follow-up.43–47 All interventions were coordinated by a designated individual, with specific training support for that role described in one.43 Case management was activated from admission46 47 or in preparation for discharge43–45 and all coordinated care following discharge.43–46 Case management interventions comprised in-hospital patient education visits,43–46 person-centred needs analysis43 44 46 and self-management strategy discussion.43 45 Key features of the postdischarge support included ongoing liaison with other specialists from acute,47 primary and community care,43–46 follow-up telephone support44–47 and dedicated telephone support for families or primary care providers.43 45
Moullec et al reported a patient-centred intervention that provided 3-hour-long self-management education sessions and ongoing case management which resulted in significantly reduced COPD-related hospitalisations over 12 months (−0.5 admissions/patient/year).45 ED presentations were not impacted but LOS was reduced from 4.0 to 3.5 days. Garcia-Aymerich et al conducted a 9-month nurse-led integrated care intervention comprising a comprehensive patient assessment and education session at discharge, development of an individually tailored plan for MDT care and ongoing telephone support.43 Patients demonstrated heightened disease knowledge, treatment adherence, nutritional status and self-management ratings.
In the study by Abad-Corpa et al, care was coordinated by two trained nurses who conducted five daily visits to eligible admitted patients to provide disease specific information, identify patient needs and liaise with other professionals, such as primary care, in preparation for discharge.44 Patient reported QoL significantly improved at 12 and 24 weeks after discharge, as did their level of knowledge about COPD. A non-significant 4% reduction in readmission rate was detected.
The discharge coordinator role presented by Lainscak et al actively involved patients and caregivers in discharge planning and communicated with community/home care services before discharge.46 In the inpatient setting, the coordinator assessed individual patient clinical and homecare needs to identify any problems and adjust in-hospital interventions. After discharge, they contacted patients by phone within 48 hours and performed a home visit after 7–10 days, while liaising with community services. Significantly fewer readmissions occurred in the intervention group (14% vs 31%).
In 2019, Gay et al reported on a pilot quality project which aimed to standardise the care provided to high risk, admitted COPD patients using automated specialist referrals, treatment checklists and coordinated care postdischarge.47 Though no improvement was found in readmissions or emergency room visits in the intervention arm, more patients attended a pulmonary follow-up visit within a month of discharge (39% vs 16%), while rates of referral to palliative care services increased with twice as many referrals to palliation in the intervention arm.
Two articles explored the implementation of AECOPD health coaching interventions.48 49 Both involved a dedicated coach who developed a partnership with patients during admission and after discharge. Education about disease management and personal health goal setting were key elements of both interventions. Wang et al indicated that levels of self-belief, self-efficacy and lung function improved over the duration of the intervention.48 Benzo et al reported a significant effect on rates of COPD hospitalisation at 1, 3 and 6 months posthospital discharge, while a significant and sustained beneficial impact on disease-specific, health-related QoL at 6 and 12 months postdischarge was observed.49 Health coaching was presented as an easily trainable and versatile intervention that can be applied to many chronic conditions.49 Both papers found their intervention increased patient confidence and their ability to manage their own conditions.
This scoping review consolidates the published evidence regarding interventions which aim to standardise care at presentation, admission and/or discharge or to reduce unnecessary admission or readmissions for patients with AECOPD. Four main intervention types were identified in the 21 studies; (1) care bundles, (2) care pathways, (3) coordinated case management and (4) health coaching interventions. Different methods of implementation were used, and varying degrees of improvement or impact were reported; with reduced LOS and readmission rate key study endpoints. Each intervention type presented opportunities for standardisation of care and MDT input, although some relied on a dedicated individual/role to encourage compliance with the intervention.
AECOPD care bundles were tested at presentation/admission or discharge. Paper-based29 30 33 or electronic format33 36 bundles were promoted to enhance standardisation of care, boost adherence to guidelines and allow opportunities to identify and rectify missed elements of care.29 QI methodology, such as stakeholder engagement and iterative service redesign and testing, was used effectively to increase adherence to bundle interventions through engagement with front-line clinicians, raising awareness and understanding of the use of care bundles.30 33 Increased compliance with bundle elements was associated with reduced LOS36 and reduced hospital readmissions.30 33
Morton et al paper noted that although clinical outcomes were not significantly improved, staff perceptions of care bundles were largely positive.35 Bundles were described a useful for standardisation of care, providing clarity around the patient pathway, facilitating effective interdisciplinary communication and identifying post-discharge support needs.50 A recently published meta-analysis of 37 studies looking at bundle implementation for treatment of various conditions echoes Morton et al’s findings of low-quality evidence, though still concluding that the implementation of care bundles may be an effective strategy to improve patient outcomes when compared with usual care.51
Epstein et al propose several key features of their tool that others might model for success including user-friendly design with prepopulation according to evidence-based guidelines, seamless integration into existing electronic resources and provision of clinical decision support to help clinicians under pressure.34 This marries well with the advice published in a 2020 review article discussing models of care in COPD; discharge bundles should be well defined, tailored to the support needs of an individual, and should be suitable for the context.50
Overall, the evidence from this review indicates that implementation of care bundles can help to ensure commonly missed elements of care are no longer missed and may enhance compliance with evidence-based treatments for AECOPD.29 Involvement of those responsible for enacting the change was found to support implementation with stakeholder meeting engagements,30 education sessions30 35 end-user feedback30 33 and mixed communication methods29 all contributing to improved implementation. Additional supports such as electronic prompts and prefilled templates30 33 34 were found to be helpful, as was nominating champions30 or offering rewards for bundle compliance.31 Patient input helped teams to understand their systems and to shape implementation in some settings. Intense bundle ‘marketing’ may also be associated with increased bundle compliance.30 Methods which use these strategies to influence behaviour change and support the implementation of care bundles should be considered by intervention teams.
All six care pathway interventions included MDT design and delivery, and all commenced from admission. All studies incorporated strategies for improved discharge planning and one specified the provision of postdischarge telephone follow-up.39 Four pathways included in-hospital patient education elements, with three conducting introductory training sessions for staff.37 38 Four pathways were overseen by a dedicated coordinator37–39 41 with the other two relying on MDT clinicians to implement the pathway.40 42
Implementation supports were not described in as much detail as those described in the care bundle studies. Vanhaecht et al used QI methods and an audit-feedback approach to help hospital teams to understand care processes within their setting, to identify gaps in care and to generate plans for improvement.38 Audit was also used by McManus et al.41 The designated coordinator role aided pathway compliance,37 39 41 however, difficulties arose if an AECOPD admission was not identified, and therefore, did not receive standardised care.39 42
Two papers noted attributed enhanced teamwork practices and reduced patient anxiety to implementation of the care pathway.40 41 Elsewhere, hospital culture and context has been linked to the likelihood of implementation success,52 improvement in care pathway processes and improved teamwork or team climate.53 There is a staffing resource implication for implementing a coordinated care pathway which may need to be considered against the degree of quantifiable benefits for clinicians and patients such as potential decreased hospitalisation costs.
Coordinated case management
The five coordinated case management interventions commence at AECOPD admission to help preparation for discharge and beyond.43–47 Interventions comprised individualised education sessions,43–46 self-management strategies43 45 and personalised case management including liaison with other services, such as social46 47 or palliative care47 and patient follow-up for between 6 and 12 months after discharge.44 46 Importantly, a focus on patient education and promotion of self-management strategies appeared to increase patient understanding of their disease,43 improve QoL44 and positively impact overall mortality.46
Studies presented elsewhere agree; coordinated case management can provide well-defined, integrated/shared-care arrangements between levels of care43 that are sustainable, person-centred and have the potential to reduce LOS54 readmission rates,55 56 mortality57 and healthcare costs.54 55 Like the care pathways, case management interventions were found to be context-dependent, requiring a tailored approach in any setting54 and should take account of individual patient needs.50 Although there are benefits to the utilisation of a designated case coordinator, the resource requirement for this type of intervention may not be feasibly replicated in other healthcare settings.
As with the integrated case management model, health coaching can be resource intensive48 due to the requirement for a dedicated, trained staff member to implement the intervention.37 48 Benefits include versatility in design as it is delivered onsite during admission and thereafter, by telephone.49 However, patients’ feelings of attention and support from the health coach may influence outcomes.48 Further, due to the multicomponent nature of the comprehensive health coaching intervention tested, the exact contributory effect of each individual component of the intervention is difficult to establish.49 Supervision of coaching sessions and use of a checklist to evaluate session content may provide support and feedback to the health coach.49
The intervention types discussed use a variety of different methodologies, with the global aim of improving AECOPD care at various stages of the AECOPD in-patient journey. Reliance on a dedicated resource may not be replicable across healthcare jurisdictions with different funding models or patient populations spanning socio-demographic boundaries. However, the benefits of investing in the standardisation of care and reducing unnecessary readmissions cannot be underestimated given the economic burden of COPD hospitalisation and impact on patient QoL. Care bundles and care pathways, when reliably implemented, have been shown to standardise care and improve care outcomes for patients with AECOPD29–42 and other chronic conditions or clinical situations.51–53 Similarly, interventions overseen by a dedicated coordinator role, whether as case manager or health coach, have had positive impact on care standardisation,47 LOS,45 readmission rate,43–46 49 and QoL,43 44 46 49 self-belief48 or satisfaction.44 The use of implementation strategies that incorporate MDT engagement and end-user education while taking account of contextual factors to enhance suitability of the intervention to the service is strongly advised. Figure 2 graphically represents these implementation supports as reported for each intervention type.
Strengths and limitations
This scoping review provides a timely summary of peer-reviewed evidence of interventions used to improve or standardise care for patients with AECOPD. Rigorous methodology was used to design, conduct and report the findings of the review. However, at the time of data extraction, little published research existed for pathway improvement interventions for AECOPD and of those, implementation methodology and outcomes were not described in granular detail, limiting the possibility of in-depth analysis. Although the variability in definition of severity of COPD exacerbation between studies is acknowledged, because the focus of this review was on the intervention being studied our findings may be generalisable to the target population. A narrow focus in the search criteria limited the number of eligible papers; geographical variation in provision of Hospital at Home and Early Discharge Support services in Ireland precluded these initiatives for review and lack of resources prevented inclusion of studies published in other languages which may have resulted in missed papers. The evidence for change in the eligible papers was further limited by small sample sizes, poor compliance with the intervention and non-statistically significant findings. In addition, sustainability of results may be contingent on the continuation of supports and active coordination of the intervention. Most studies tended not to include economic impact of the intervention although Morton et al found no evidence for cost savings after bundle implementation.35 Others noted potential for cost savings through reduced LOS40 or readmissions.48 Future studies are recommended to examine this aspect of improvement outcomes.
The aim of this review was to seek primary evidence from existing literature relating to improvement interventions which seek to achieve better outcomes such as standardisation of care at presentation, admission and/or discharge and reduction in unnecessary admission/readmission rates for patients with AECOPD. This summary provides evidence of a supportive approach for policy-makers, planners and medical practitioners in designing implementation supports for testing new interventions. Though a number of the studies reported no significant change in the primary stated outcome of reduced LOS, readmissions or use of health services, most saw a trend in improved outcomes in their intervention populations including person-centred elements such as patient confidence and understanding of their disease, and staff adherence to bundle interventions. Several studies presented here state that engagement with the front-line staff-users of the intervention, whether bundle or pathway, enhances sustainability of improvements. When designing a new improvement initiative for AECOPD care, consideration of the implementation strategy and methods of support will be necessary. In the Irish context, QI Collaborative methodology will be adapted to work with up to 20 front-line teams across Ireland to use the Model for Improvement and PDSA cycles to design and test bespoke local service improvements that reflect national strategic priorities of standardised, evidence-based AECOPD care.
Contributors Thank you to all contributors to this paper. RM as lead author, OW, SW and LP as research associates. All reviewers; TM, BC, JB, EK, AC, KB, NM and AL; contributed to the study selection and draft review process. RM, OW, SW, LP and TM read and approved the final manuscript.
Funding The National COPD Collaborative in Ireland is funded by Clinical Strategy and Programmes, Health Service Executive Ireland.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement No data are available.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.