1. Background

1.1 Joint Commission International Accreditation

The Joint Commission International (JCI) extends the Joint Commission's mission and standards worldwide by helping international healthcare organizations improve patient care quality and safety. The Joint Commission originated from the American College of Surgeons (ACS), founded in 1910 [1]. According to the history of the Joint Commission, the American College of Physicians, the American Hospital Association, the American Medical Association, and the Canadian Medical Association joined with ACS to create the Joint Commission on the Accreditation of Hospitals (JCAH) in 1950. JCAH was renamed the Joint Commission on the Accreditation of Healthcare Organizations (JCAHO) to reflect its expanded scope of services in 1987. JCAHO shortened its name to the Joint Commission in 2007. JCAHO formed JCI to provide international clients with education and consulting services in 1994. JCI published its first international quality standards for hospitals in 2000 [2]. As of December 2022, 946 healthcare organizations had obtained JCI accreditation worldwide [3].

In total, there were 36,570 hospitals, including 11,804 public hospitals and 24,766 private hospitals, in China at the end of 2021. Public hospitals are significant providers, and they served 83.9% of outpatients and 81.5% of inpatients in China from January to April 2022 [4]. Public hospitals, sponsored by the Chinese government, are not-for-profit healthcare organizations. However, most private hospitals are for-profit organizations whose shareholders and investors expect investment returns. As of December 2022, 44 Chinese hospitals had obtained valid JCI accreditation, and private hospitals accounted for 93.2% (41/44) of all accredited hospitals. General and OBGYN (obstetrics and gynecology) hospitals accounted for 45.5% (20/41) and 27.3% (12/41) of all private accredited hospitals, respectively. The other nine private accredited hospitals are specialty hospitals in the fields of plastic surgery (six), pediatrics (one), neurology (one), and rehabilitation (one) [3]. Those hospitals lose accreditation when they fail to participate in the JCI reaccreditation survey every three years after successfully passing the initial JCI accreditation survey.

1.2 Literature Review on the Impact of Hospital Accreditation

Hospital accreditation is an external, independent assessment of healthcare organizations based on accepted standards. An accreditation program should be a significant driver of improvements to the quality and safety of healthcare organizations. However, critics question the value and impact of accreditation [5]. Empirical studies on the effects of hospital accreditation conducted worldwide have shown inconsistent findings. In one study, the authors found that hospital accreditation improves care processes and outcomes. The authors pointed out that accreditation could be a tool to enhance the quality of care [6]. Other studies have concluded that hospital accreditation helps improve safety culture, process-related performance, and efficiency [7] and [8]. However, some researchers have failed to find robust evidence supporting a causal inference between hospital accreditation and measurable changes in the quality of care. They have argued that hospital accreditation programs distract healthcare professionals from their primary clinical goals and burden them with financial and labor costs [9]. The effects of JCI accreditation on the quality of care and performance have been assessed in different countries since 2010. In one study, the author concluded that pursuing JCI accreditation positively impacted 75% (9/12) of a select list of clinical measures at the 550-bed King Fahd Hospital in Saudi Arabia [10]. In another study, 60% (3/5) of the studied performance indicators achieved significant improvements due to JCI accreditation; this was found by comparing two 133-bed accredited hospitals and two 115-bed unaccredited hospitals in Jordan [11]. It was also determined that JCI accreditation helped improve operating room efficiency for patients under general anesthesia [12] and topical anesthesia at Juntendo University Hospital in Japan [13]. In a study of a JCI-accredited 150-bed acute care hospital in the United Arab Emirates, researchers raised the concern that it is challenging to sustain improved outcomes. They found an immediate drop in performance after completing the JCI accreditation survey. During the post-accreditation period, only 4% (1/27) of the selected quality measures showed a significant positive trend, but 48% (13/27) showed substantial negative changes [14]. In their comparative study of a JCI-accredited 650-bed tertiary academic hospital in the United Arab Emirates, these authors concluded that participating in reaccreditation surveys is the solution to continuous improvement [15].

2. Methods

A JCI-accredited hospital should comply with JCI standards, usually updated every three years, regarding new healthcare practices, quality management, and technology. JCI accreditation is an intervention for healthcare organizations. Most studies on the impact of JCI accreditation have conducted qualitative in-depth interviews, cross-sectional surveys, and statistical regression analyses.

2.1 Study Design

Treating JCI accreditation as an intervention, we assessed the relationship between the impact of JCI accreditation and the selected clinical, financial, and operational performance measures in Chinese private hospitals. When randomization is impossible, ITSA can be used as a robust quasi-experimental study approach to evaluate an intervention's effects in a longitudinal dataset [16]. ITSA focuses on quality improvement and requires a minimum of eight observations (data points) before and after the intervention to identify the change in an outcome between the periods before and after the intervention [17]. One of the strengths of ITSA is that it is unaffected by typical confounding variables. To deal with time-varying confounders that change relatively rapidly, researchers can add a control group to the ITSA [18]. ITSA has a high degree of internal validity, given adequate observations of an outcome variable. Researchers can further enhance this internal validity by comparing the treated group's outcomes with those of one or more control groups [19]. With one or more control groups, ITSA is more flexible and powerful than difference-in-difference (DID) methods [20]. Given the characteristics of the study population and the number of retrospective observations, a multiple-group ITSA is an appropriate design for our study. In the studies cited in the paragraph, researchers treated JCI accreditation as an intervention and used quasi-experimental approaches, such as ITSA and DID methods, to assess the impact of JCI accreditation on selected measures.

2.2 Study Population

Our study was conducted in two 200-bed private hospitals, Kunming Angel OBGYN Hospital (Hospital A) and Xi'an Angel OBGYN Hospital (Hospital B). Hospitals A and B are comparable because they are subsidiaries of the same OBGYN specialty hospital group (the parent company) and provide the same medical services in China. Hospital A began operating in 2013, and Hospital B began operating in 2014. The chief executive officer of the parent company appoints the presidents and vice presidents of Hospitals A and B, which apply the same administration system. Hospitals A and B have the same brand strategy and similar marketing tactics even though they are in different cities. From the perspective of the addressable market, Hospitals A and B are comparable in terms of the city population and residents' income. We treated Hospital A as the intervention group and Hospital B as the comparison group because Hospital A participated in the initial JCI accreditation survey in 2014 and re-accreditation surveys in 2017 and 2020. However, Hospital B decided not to join JCI reaccreditation after passing the initial JCI survey in 2015.

2.3 Data source and study variables for performance measures

The eight outcome measures of clinical, financial, and operational performance are shown in Table 1. The clinical quality measures used in the Chinese OBGYN specialty hospital were the C-section rate (Y1), the perineal incision rate (Y2), the incidence of macrosomia (Y3), and the preterm birth rate (Y4). The operational performance indicators were the number of outpatient visits (Y5) and the number of deliveries (Y6). The financial performance measures were revenue (Y7) and earnings before interest, tax, depreciation & amortization (EBITDA) (Y8). We compared Hospital A and Hospital B's performance across monthly intervals between the preintervention (from January 2015 to December 2015) and postintervention periods (from January 2016 to December 2022).

Table 1:  Clinical, operational and financial performance measure descriptions

2.4 Data Analysis

In the following regression equation, β_0 to [ β]_3 represent the comparison group, and β_4 to [ β]_7 represent the intervention group. The critical assumption in multiple-group ITSA is that the confounding variables affect both the intervention and comparison groups similarly [19].

Y_t = + + + + + + + +e_t

where:

i. Yt: the outcome variable at time t;

ii. Tt: the time in months from the start of the study period to the last time point in the series, representing the frequency of monthly observations;

iii. Xt: a dummy variable representing the preintervention period (coded as "0") or postintervention period (coded as 1);

iv. Z: a dummy variable indicating the intervention group (coded as "1") or the comparison group (coded as "0");

v. β_0: the intercept (the baseline level at T=0 in the comparison group);

vi. β_1: the slope before the intervention (the change in outcome associated with a time unit increase representing the underlying preintervention trend);

vii. β_2: the level change in the period immediately following the intervention;

viii. β_3: the slope change between the preintervention and postintervention periods (using the interaction XT between the intervention and time)

ix. β_4: the difference in the intercept between the intervention and comparison groups before the intervention;

x. β_5: the difference in the slope between the intervention and comparison groups before the intervention;

xi. β_6: the difference in the level between the intervention and comparison groups in the period immediately following the intervention;

xii. β_7: the difference in the slope between the intervention and comparison groups after the intervention compared to that prior to the intervention.

We aimed to assess the associations between the independent variable X, JCI accreditation, and the dependent variable Y, representing clinical, financial, and operational performance. We used STATA 17.0 to conduct multiple-group ITSA to compare the postintervention trends in the outcome variables between Hospital A (the intervention group) and Hospital B (the comparison group). The study period spanned eight years, including the pre-intervention period from January 2015 to December 2015 and the post-intervention period from January 2016 to December 2022. The scatter plot of monthly observations against time revealed certain data features, including trends, seasonality, outliers, and turning points, as shown in Figures 1-4. Hospitals A and B followed the same organizational governance and marketing strategies during the study period. We assumed the JCI accreditation intervention was the critical event impacting the time series.

Figure 1: Monthly C-Section Rate; January 2015 to December 2022.

Figure 2: Monthly Outpatient Visits; January 2015 to December 2022.

Figure 3: Monthly Deliveries; January 2015 to December 2022.

Figure 4: Monthly Revenue; January 2015 to December 2022.

3. Results

3.1 Descriptive Statistics

Data were collected for 96 months between January 2015 and December 2022 in each hospital with no gaps. The mean, median, standard deviation (SD), min (the lowest value), max (the highest value), and percentiles (p5, p25, p75, and p95) of the dependent variables are depicted in Table 2. The data were symmetrical because the measures' means and medians were similar.

Table 2:  Summary statistics of monthly observation over eight years from Jan 2015 to Dec 2022

3.2 Regression Statistics

We used STATA 17.0 to obtain the results shown in Table 3 and Figures 1-4, which show the levels and trends of Hospitals A and B. Table 3 displays the regression statistics of the time series before and after the intervention in January 2016 for the dependent variables corresponding to Hospital A and Hospital B. Lower values of the clinical measures, such as the C-section rate, perineal incision rate, incidence of macrosomia, and preterm birth rate, indicates better clinical quality. A higher number of OP visits or deliveries shows better operational performance. Higher revenue or EBITDA indicates better financial performance. P value of <0.05 was used for statistical significance.

Table 3:  Time series analysis for the eight performance measures

The C-section rate is a critical indicator used to measure the quality of care in Chinese OBGYN hospitals. A decreasing trend in the C-section rate implies continuous improvement. Before the intervention, Hospital B showed a downward trend (β1=-2.6%, P<0.001, 95% CI [-3.9%, -1.2%]). However, immediately following the intervention time point, when Hospital B decided not to participate in the JCI accreditation survey, the C-section rate downward trend became slower (β3=2.5%, P<0.001, 95% CI [1.2%,3.9%]). Compared with the change in the downward trend of Hospital B, the downward trend of Hospital A became faster after the intervention (β5=1.9%, P=0.024, 95% CI [0.2%, 3.5%]; β7=-2%, P=0.016, 95% CI [-3.7%, -0.4%]). Moreover, the JCI accreditation intervention had a significant association with the C-section rate, as illustrated by the decreasing trend in Figure 1. However, the results did not show a statistically significant impact of JCI accreditation on the other clinical performance measures, including the perineal incision rate, the incidence of macrosomia, and preterm birth rate, in our study.

We hypothesized that JCI accreditation to increase the number of outpatient visits and the number of deliveries, leading to improved financial performance. The outpatient visits of Hospitals A and B increased during the pre-and postintervention periods. As seen in Figure 2, monthly outpatient visits grew faster in Hospital A than in Hospital B. Notably, in Hospital A, monthly outpatient visits increased by 102.492 visits (P<0.001, 95% CI [88.529, 116.455]), and in Hospital B, they increased by only 56.946 visits (P<0.001, 95% CI [50.676, 63.215]) after the intervention. The difference between the hospitals (Hospital A minus Hospital B) was 45.547 outpatient visits per month (P<0.001, 95% CI [30.241,60.853]). Regarding the impact of JCI accreditation on the growth trends in monthly deliveries, the trend of deliveries in Hospital B was worse in the postintervention period than it was in the preintervention period (β1=2.451, P<0.001, 95% CI [1.945, 2.957]; β3=-2.070, P<0.001, 95% CI [-2.623, -1.518]). Compared with the trend of Hospital B, Hospital A's trend improved after the intervention (β5=-3.294, P<0.001, 95% CI [-4.966, -1.622]; β7=3.090, P<0.001, 95% CI [1.408, 4.773]). Figure 3 displays these trends. The economic impact of JCI accreditation on monthly revenue (US$ million) could be quantified as the estimated coefficient of the difference (coefficient=0.014, P<0.001, 95% CI [0.009, 0.019]) between Hospital A and Hospital B after the intervention. Hospital A generated US$14,000 more in revenue over the postintervention period than Hospital B. And the estimated coefficient of the difference in monthly EBITDA (coefficient=0.004, P=0.019, 95% CI [0.0006, 0.0067]) indicated that Hospital A could generate more US$ 4,000 of monthly EBITDA over the postintervention period than Hospital B. Figure 4 displays the comparison trends after the intervention.

4. Discussion

This study is the first to evaluate the impact of JCI accreditation on Chinese private hospitals. Maintaining a hospital's accreditation requires resources such as money and time; whether a hospital's accreditation is associated with a measurable improvement in its performance is critical for healthcare policymakers and hospital management to know who aims to support investments in the accreditation program. JCI accreditation is designed to evaluate healthcare organizations' full range of functions. The systems and processes supporting patient care and leadership in a JCI-accredited hospital comply with JCI standards, which are typically updated every three years based on new healthcare practices, quality management methods, and technology guidance. We hypothesized that JCI accreditation results in better clinical outcomes and financial and operational performance. Hospital A engaged in three JCI accreditation surveys from 2014 to 2020, but Hospital B participated in only the JCI accreditation survey in 2015. Some researchers found that the number of years that a hospital has been engaged in hospital accreditation can affect organizational changes that enhance the quality of care [21]. Our results showed a significant association between the C-section rate and JCI accreditation. JCI accreditation had no statistically significant impact on the other selected clinical performance measures. This evidence confirms findings from previous studies conducted in other countries. Some studies did not find strong evidence of a relationship between hospital accreditation and specific quality indicators [22]. Other studies found a complex relationship when comparing the indicator performance of accredited and unaccredited hospitals. The accredited hospitals performed better than the unaccredited hospitals on some quality indicators but failed to perform better on others [23]. In an updated systematic review, authors found positive, negative, and no association between hospital accreditation and organizational performance [24]. JCI accreditation could be used as a marketing tool, as it may reassure patients and enhance their confidence in the high quality of care and safety [22]. Hospital accreditation requires external assessments because patients cannot observe the quality of healthcare services benchmarked against standards and processes [25]. Our results showed that Hospital A performed better in increasing revenue, outpatient visits, and deliveries than Hospital B during the postintervention period. We think the gold seal of JCI accreditation could cause more patients to visit an accredited hospital than a comparable unaccredited hospital because of the JCI's reputation for assessing excellence in the quality of care and patient safety. Hospital A achieved a better EBTIDA trend than Hospital B during the postintervention period. One reason is that more revenue would improve financial margin given the fixed cost structure.

4.1 Implications

The management of Hospital B may argue that they have learned the JCI standards even though they decided not to participate in the JCI survey after passing the initial survey. Most Chinese private hospitals are for-profit hospitals. Only when a for-profit hospital believes its revenue exceeds the costs of obtaining an accreditation will the hospital be motivated to seek the accreditation [25]. For Chinese healthcare policymakers, if there is evidence showing that the JCI standards help improve the quality of care and patient safety, foreign hospital accreditation can complement local accreditation. Additionally, Chinese private hospital management may use JCI accreditation as a management tool to drive integration, collaboration, and improvement. When a private hospital is newly set up based on various teams with different backgrounds, the accreditation process might accelerate the integration of clinical procedures and promote a safety culture. In addition, there may be value in participating in the JCI accreditation survey process since the associated external assessment promotes the continuous improvement of care quality.

4.2 Limitations

The study results cannot be generalized to Chinese public hospitals, but generally, Chinese public hospitals have far fewer motivations than private hospitals to pursue JCI accreditation. The evidence provided in our study stems from OBGYN specialty hospitals, so the conclusions may not be generalized to other types of hospitals. Also, the study findings might not be generalized to other OBGYN hospitals either. Although our quantitative outcome-based analysis may examine the value of JCI accreditation, it is not easy to evaluate the challenges related to accreditation. We recommend that further research be conducted to explore the sustainability of JCI accreditation within Chinese private hospitals. The comparison group was exposed to the JCI program. Our study could not differentiate between the impact of such exposure on the comparison group's performance and the effects of JCI reaccreditation on the intervention group.

5. Conclusions

Over 90% of JCI-accredited Chinese hospitals are private hospitals. This paper is the first study to use a multiple-group time series analysis to evaluate the impact of JCI accreditation on clinical, operational, and financial performance in Chinese private hospitals. The evidence we obtain by comparing Hospitals A and B demonstrates that the subsequent JCI accreditation surveys improve one of the four selected clinical indicators and increase outpatient visits, deliveries, and revenue. Therefore, we conclude that although JCI accreditation may not be significantly associated with the improvement of all the select performance measures, it does seem to be associated with continuous improvement in accredited Chinese private hospitals.

List of abbreviations

ACS: American College of Surgeons

JCAH: Joint Commission on the Accreditation of Hospitals

JCAHO: Joint Commission on the Accreditation of Healthcare Organizations

JCI: Joint Commission International

ITSA: Interrupted Time Series Analysis

OBGYN: obstetrics and gynecology

OP: Outpatient

DID: Difference-In-Difference

EBITDA: Earnings before interest, tax, depreciation & amortization

SD: Standard Deviation

Declarations

Ethics Approval and Consent to Participate

Not applicable

Consent for Publication

Not applicable

Availability of Data and Materials

The dataset supporting the conclusions of this article is included within the article.

Competing Interests

The authors declare that they have no competing interests.

Funding

Not applicable

Authors' Contributions

HZ conducted the study and drafted the manuscript under MJB’s supervision. HCC and STH participated in the study design. All authors read and approved the final manuscript.

Acknowledgments

Not applicable

References

1. Bord S, Sass I, Hayms G, et al. Involvement and skepticism towards the JCI Accreditation process among hospital's four sectors employees: suggestions for cultural change. Isr J Health Policy Res 10 (2021): 74.
2. The Joint Commission. The Joint Commission History Timeline. In: History of the Joint Commission (2022).
3. The Joint Commission International. Country/Region/Territory: China. In: Search for JCI-accredited organizations (2022).
4. National Health Commission of the People's Republic of China. 2021 China's health development statistical bulletin. China: National Health Commission of the People's Republic of China (2022).
5. Shaw CD, Kutryba B, Braithwaite J, et al. Sustainable healthcare accreditation: messages from Europe in 2009. Int J Qual Health Care 22 (2010): 341-50.
6. Alkhenizan A, Shaw C. Impact of accreditation on the quality of healthcare services: a systematic review of the literature. Ann Saudi Med 31 (2011): 407-16.
7. Araujo CAS, Siqueira MM, Malik AM. Hospital accreditation impact on healthcare quality dimensions: a systematic review. Int J Qual Health Care 32 (2020): 531-44.
8. Hussein M, Pavlova M, Ghalwash M, et al. The impact of hospital accreditation on the quality of healthcare: a systematic literature review. BMC Health Serv Res 21 (2021): 1057.
9. Brubakk K, Vist GE, Bukholm G, et al. A systematic review of hospital accreditation: the challenges of measuring complex intervention effects. BMC Health Serv Res 15 (2015): 280.
10. Al Shawan D. The effectiveness of the joint commission international accreditation in improving quality at king fahd university hospital, Saudi Arabia: a mixed methods approach. J Healthc Leadersh 13 (2021): 47-61.
11. Halasa YA, Zeng W, Chappy E, et al. Value and impact of international hospital accreditation: a case study from Jordan. East Mediterr Health J 21 (2015): 90-9.
12. Inomata T, Mizuno J, Iwagami M. The impact of joint commission international accreditation on time periods in the operating room: A retrospective observational study. PloS One 13 (2018).
13. Okumura Y, Inomata T, Iwagami M, et al. Shortened cataract surgery by standardisation of the perioperative protocol according to the Joint Commission International accreditation: a retrospective observational study. BMJ Open 9 (2019): 028656.
14. Devkaran S, O'Farrell PN. The impact of hospital accreditation on quality measures: an interrupted time series analysis. BMC Health Serv Res 15 (2015): 137.
15. Devkaran S, O'Farrell PN, Ellahham S, et al. Impact of repeated hospital accreditation surveys on quality and reliability, an 8-year interrupted time series analysis. BMJ Open 9 (2019):
16. Kontopantelis E, Doran T, Springate DA, et al. Regression based quasi-experimental approach when randomisation is not an option: interrupted time series analysis. BMJ 350 (2015): 2750.
17. Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr 13 (2013): 38-44.
18. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol 46 (2017): 348-55.
19. Linden A. Conducting interrupted time-series analysis for single- and multiple-group comparisons. Stata J 15 (2015): 480-500.
20. Bernal JL, Cummins S, Gasparrini A. Difference in difference, controlled interrupted time series and synthetic controls. Int J Epidemiol 48 (2019): 2062-3.
21. Pomey MP, Lemieux-Charles L, Champagne F, et al. Does accreditation stimulate change? A study of the impact of the accreditation process on Canadian healthcare organizations. Implement Sci 5 (2010): 31.
22. Braithwaite J, Greenfield D, Westbrook J, et al. Health service accreditation as a predictor of clinical and organisational performance: a blinded, random, stratified study. Qual Saf Health Care 19 (2010): 14-21.
23. Greenfield D, Braithwaite J. Health sector accreditation research: a systematic review. Int J Qual Health Care 20 (2008): 172-83.
24. Hinchcliff R, Greenfield D, Moldovan M, et al. Narrative synthesis of health service accreditation literature. BMJ Quality and Safety 21 (2012): 979–991.
25. Grepperud S. Is the hospital decision to seek accreditation an effective one? Int J Health Plann Manage 30 (2015): 56-68.