Using the Elixhauser risk adjustment model to predict outcomes among patients hospitalized in internal medicine at a large, tertiary-care hospital in Israel

Abstract

Background

In Israel, internal medicine admissions are currently reimbursed without accounting for patient complexity. This is at odds with most other developed countries and has the potential to lead to market distortions such as avoiding sicker patients. Our objective was to apply a well-known, freely available risk adjustment model, the Elixhauser model, to predict relevant outcomes among patients hospitalized on the internal medicine service of a large, Israeli tertiary-care hospital.

Methods

We used data from the Shaare Zedek Medical Center, a large tertiary referral hospital in Jerusalem. The study included 55,946 hospitalizations between 01.01.2016 and 31.12.2019. We modeled four patient outcomes: in-hospital mortality, escalation of care (intensive care unit (ICU) transfer, mechanical ventilation, daytime bi-level positive pressure ventilation, or vasopressors), 30-day readmission, and length of stay (LOS). We log-transformed LOS to address right skew. As is usual with the Elixhauser model, we identified 29 comorbid conditions using international classification of diseases codes, clinical modification, version 9. We derived and validated the coefficients for these 29 variables using split-sample derivation and validation. We checked model fit using c-statistics and R², and model calibration using a Hosmer–Lemeshow test.

Results

The Elixhauser model achieved acceptable prediction of the three binary outcomes, with c-statistics of 0.712, 0.681, and 0.605 to predict in-hospital mortality, escalation of care, and 30-day readmission respectively. The c-statistic did not decrease in the validation set (0.707, 0.687, and 0.603, respectively), suggesting that the models are not overfitted. The model to predict log length of stay achieved an R² of 0.102 in the derivation set and 0.101 in the validation set. The Hosmer–Lemeshow test did not suggest issues with model calibration.

Conclusion

We demonstrated that a freely-available risk adjustment model can achieve acceptable prediction of important clinical outcomes in a dataset of patients admitted to a large, Israeli tertiary-care hospital. This model could potentially be used as a basis for differential payment by patient complexity.

Introduction

Patients admitted to an internal medicine (IM) service in a hospital can be quite complex. By definition, these patients are ill enough to warrant hospitalization. Many have a significant list of chronic comorbidities, take multiple medications on an ongoing basis, or may have other complicating factors that are not consistently captured in structured data. For these reasons, it can be a challenge to define the complexity of patients admitted to internal medicine [1].

This difficulty can be reflected in billing practices. It is more straightforward and usually more lucrative to bill for admissions involving procedural interventions rather than for IM patients being treated for multiple non-surgical conditions. In Israel, reimbursement of hospital care varies. Inpatient procedures are paid for either per diem or through activity-based procedure-related group (PRG) arrangements, which usually pertain to the activities of surgical or procedure oriented specialties [2]. Until 2022, care provided by the IM service was reimbursed on a per-diem basis by the health plan—that is, a flat fee per patient per night. Starting in 2023, hospitals have been reimbursed through a prospective annual budget based on the previous year’s utilization. Neither of these mechanisms reflects the patient’s comorbidities or the severity of his or her clinical condition. Thus, hospitals that take care of more complex patients do not receive any extra payment.

Adjusting for complexity is important, to ensure that hospitals that treat more complicated patients receive adequate reimbursement [3]. The usual way to adjust for differential complexity between patients is to apply a risk adjustment model, also known as case mix adjustment [3]. There are many existing risk adjustment models that can be used. The idea of using them is that they account for a group of measured variables that together define how complex a patient is likely to be. These models have been shown to be very predictive of outcomes such as in-hospital mortality and length of stay, and somewhat less predictive of readmissions (which are very hard to predict) [4, 5]. Some risk adjustment models are freely available, but in fact most are proprietary. In addition to using them as a basis for differential payment to hospitals, they are also used to adjust severity in observational research studies, and by hospitals to support their operations and planning.

In most countries, some sort of risk adjustment model is applied to support differential payment for hospital care. Without such adjustment, hospitals that care for more complex patients are burdened with inadequate resources. Beginning in 1983, the US Centers for Medicare and Medicaid Services (CMS) adopted diagnosis-related groups (DRGs) as a way to group hospitalized patients by severity and expected complexity of care. The existence of DRGs was part of what enabled hospitals to reduce lengths of stay and reduce costs over time [6]. Since that time, there has been continued innovation in terms of grouping patients more precisely. For example, CMS has developed a more advanced grouping system (Medicare Severity Diagnosis Related Groups, or MS-DRG) to determine hospital reimbursement rates. Their index reflects the diversity, complexity, and severity of a patient’s illness at the time of admission. Other developed have also innovated in terms of making the DRG concept more precise over time to support differential payment by complexity [7, 8].

In Israel, there is not currently a system of case mix adjustment to support differential payment for hospital care. Here, we will demonstrate that a publicly-available system can be used successfully in an Israeli context. It is logical that we would use a publicly available case mix model rather than a proprietary one. Two popular models for estimating patient complexity are the Charlson Comorbidity Index (CCI) and the Elixhauser Comorbidity Index (ECI). The CCI is a composite score summarized by a weighted combinations of 17 comorbidities, which was originally developed to predict the risk of death among breast cancer patients [9], and has been adapted for use with administrative datasets [10, 11]. The original Elixhauser system measures 30 dichotomous variables [12], and has also been adapted for use in hospital-based case mix adjustment.

Both of these methods can be used to predict mortality, length of stay, readmission, and heath care utilization [13, 14]. Several studies have demonstrated Elixhauser’s system to be superior for predicting various outcomes [15,16,17]. The Elixhauser Comorbidity Index has not been used in Israel to try and define and evaluate the burden of comorbidities for patients admitted to IM departments.

The aim of this study was to apply the ECI to patients admitted to a tertiary-care center in Israel for internal medicine care, to more fully reflect the complexity and severity of the patient population treated there. We examine the ability of the model to predict the outcomes of in-hospital mortality, need for intensive care, 30-day readmission, and length of stay. We expected to show that this model could predict these outcomes here as it has elsewhere. Demonstrating the validity of this model for use in Israeli IM patients could support billing reform and help reshape the landscape in terms of reimbursement for inpatient IM services.

Methods

Database

We used data from the Shaare Zedek Medical Center (SZMC), a large tertiary-care referral hospital in Jerusalem that serves a varied population in terms of ethnicity and socioeconomic status. The study included 55,946 hospitalizations between 01.01.2016 and 31.12.2019. These dates were chosen to allow us to study care under usual conditions, prior to the influence that coronovirus disease 2019 (COVID-19) had on the Israeli medical system. The study unit was hospitalizations rather than patients, since some patients were admitted more than once. Another reason to focus our study on hospitalizations as the unit of analysis, rather than unique patients, is that case mix models are in part intended to support differential payment by complexity for hospitalizations, rather than for unique individuals per se.

For the purposes of this study, “internal medicine wards” were defined as the four formal IM departments (A, B, C, and D), geriatrics, cardiology, and a short stay unit. The rationale for including these additional units is that some Israeli hospitals do not have such units, and therefore the patients hospitalized at SZMC in these wards would be part of the population served by IM in a different hospital setting. In order to capture the entire spectrum of IM patients, they are included here. This study was approved by the research ethics committee of SZMC (0361-21-SZMC).

Demographics and patient characteristics

Patient deidentified medical information was extracted from the hospital's electronic medical record. We characterized demographics, dates of hospitalization and discharge, date of birth, gender, and all wards the patient visited during the hospital stay.

Dependent variables: patient outcomes

There were four dependent variables for this study, each of which was modeled as a separate outcome. Three are binary outcomes, and the last is a continuous outcome. The first outcome was in-hospital mortality. The second was readmission to the hospital within 30 days of the date of discharge. Because our data are limited to SZMC, readmissions to other hospitals would not be captured, but SZMC patients are primarily readmitted back to SZMC [18].

The third outcome was requiring an escalation of care beyond the regular medicine ward. Our definition of this outcome was expansive, and included any patient who spent part of their hospital stay in the intensive care unit (ICU), the intermediate care unit (IMCU i.e., a monitored setting, part of the IM ward, but with fewer patients per nurse), who received mechanical ventilation, daytime bi-level positive pressure ventilation (BiPAP), or vasopressors (i.e., medications intended to support blood pressure). Daytime BiPAP was defined as occurring between 8 AM and 8 PM. BiPAP received at night may be needed for disordered breathing during sleep (i.e., obstructive sleep apnea), but BiPAP received during the day is presumably intended as a method to prevent invasive mechanical ventilation. Vasopressors included adrenaline, dobutamine, dopamine, milrinone, noradrenaline, phenylephrine, and vasopressin. Any of these interventions (i.e., ICU, IMCU, mechanical ventilation, daytime BiPAP, or vasopressors) sufficed to show that the patient had at least some degree of critical illness and required intensive intervention. Our group has previously published about the relationship between bed location and the receipt of mechanical ventilation, daytime BiPAP, and vasopressors [19]. Some patients experienced an escalation of care, and then died during the same hospitalization. Such patients were included in both analyses.

The fourth outcome, which was continuous, was the length of stay (LOS), measured in days. Because LOS is known to be right-skewed (i.e., not normally distributed), we performed a log transformation, which did result in a normal distribution. As such, we report outcomes for the log-LOS—as is usual in many studies [20].

Independent variables: Elixhauser index

The Elixhauser Index is a case-mix adjustment model based on patient comorbidities that predicts outcomes such as our four outcomes. Originally developed by Elixhauser and colleagues, it has since been used as a case-mix adjustment model by other investigators, such as Van Walraven and colleagues [21]. The original Elixhauser paper gives a list of diagnosis codes that are used to define 30 comorbid conditions, in the International Classification of Diseases, Version 9 (ICD-9). The codes, and the diagnoses they stand for, are in Table 1, and are taken from the original Elixhauser paper.

Table 1 Definitions of comorbidities

At SZMC, as in the entire Israeli health system, diagnoses are recorded using the ICD-9 system, which is compatible with this list of codes. Patients were defined as having a diagnosis if they recorded a code for it during their hospitalization. Each hospitalization was analyzed as a separate unit, so codes recorded for the same patient during a different hospitalization did not count. In using the codes, we noticed that only 6 patients had the diagnosis of iron deficiency anemia as defined by the Elixhauser model; perhaps this set of codes is not often used at SZMC, and it is likely that they are grouped instead with the “deficiency anemias” category. Because we expected such a small number to interfere with our models due to small cell counts, we deleted this condition and based our analyses on the other 29 conditions.

Statistical analyses

We began by examining the frequency of basic information (e.g., sex, age, etc.) and the 29 comorbid conditions in the Elixhauser model. We also constructed our four outcome variables and examined their frequencies in the dataset.

When using the Elixhauser model, one is meant to derive the model coefficients anew each time. This means that the beta coefficients for each of the 29 conditions are fit to the particular dataset. In fact, we had four sets of coefficients for the model, one for each of the outcomes. We randomly divided our dataset 70/30 into derivation and validation sets. We used the derivation set to derive our model coefficients for each of the outcomes, and then forced those coefficients onto the validation set to check that model fit did not decrease (which would have indicated overfitting). For binary outcomes, which were modeled using a logistic model, model fit was characterized using the c-statistic, also known as the area under the receiver operator curve (ROC). For the continuous outcome (log LOS), which we modeled using an ordinary least squares (OLS) model, fit was characterized using R².

We also characterized model calibration, which can be thought of as a measure of whether the model systematically overestimates risk in lower-risk or in higher-risk groups, and by how much. For the logistic models, we did this using a Hosmer–Lemeshow test [22]. There is not a clearly analogous test for a linear model. Analyses were performed using SPSS version 24 and R Studio version 1.3.1093.

Results

Dataset

The baseline characteristics for the 55,946 hospitalizations during the study period can be seen in Table 2. The unit for our analysis was hospitalization, and as such, some patients appear more than once in the data. Most admissions (86%) were via the Emergency Room, with the remainder being elective admissions or transitions between departments within the hospital. Elective admissions to the internal medicine service are very rare; therefore, the great majority of patients admitted electively were initially admitted to surgery, then transferred to internal medicine later. The median age of the patients was 74 years, and most (53%) patients were male.

Table 2 Baseline characteristics of hospitalizations from 2018–2019 (N = 55,946)

Patient outcomes

Approximately 7% of the hospitalizations resulted in patient deaths, 13% involved an escalation of care, and 15% of the total admissions resulted in readmission within 30 days. The median LOS was 4.1 days (IQR 2.1, 7.8).

Balance between derivation and validation sets

We randomly divided the sample into a derivation set (39,162 hospitalizations, or 70%) and a validation set (16,784 hospitalizations, or 30%). Table 3 shows the balance that we achieved across the derivation and validation sets regarding the prevalence of Elixhauser comorbid conditions. Hypertension and congestive heart failure were the most common comorbid conditions. The derivation and validation sets were balanced in terms of the prevalence of these comorbid conditions, and thus in the severity of their comorbid illness burden (and in their overall Elixhauser risk).

Table 3 Balance table

Model performance

The multivariate regression for predicting outcomes using the Elixhauser model derivation set is displayed in Table 4. The three binary outcomes are predicted using logistic regression, while log LOS is predicted using a linear regression. Because LOS is not normally distributed, we log transformed it for the linear regression, as is customary.

Table 4 Multivariate regression of the derivation set (n = 37, 690)

The Elixhauser model achieved acceptable prediction of the three binary outcomes, with c-statistics of 0.712, 0.681, and 0.605 to predict in-hospital mortality, escalation of care, and 30-day readmission respectively (Fig. 1). The c-statistic did not decrease in the validation set (0.707, 0.687, and 0.603, respectively), suggesting that the models are not overfitted.

Fig. 1

Receiver operator curves for the derivation set (column A) and the validation set (column B)

The most predictive conditions for in-hospital mortality were fluid and electrolyte disorders (OR = 2.52, 95% CI 2.29, 2.77; for this and all other results discussed here p < 0.001, unless otherwise stated), coagulopathy (OR = 2.49, 95% CI 2.04, 3.03), and metastatic cancer (OR = 2.45, 95% CI 2.03, 2.95). The most predictive conditions for escalation of care were fluid and electrolyte disorders (OR = 2.18, 95% CI 2.02, 2.35) and other neurological disorders (OR = 1.96, 95% CI 1.73, 2.22). The acquired immunodeficiency syndrome (AIDS) was highly predictive when present (OR = 4.02, 95% CI 0.95, 14.4; p-value = 0.039), but impacted very few people. The most predictive conditions for 30-day readmission were weight loss (OR = 1.67, 95% CI 1.20, 2.29) solid tumor without metastasis (OR = 1.56, 95% CI 1.42, 1.72) and psychoses (OR = 1.51, 95% CI 1.28, 1.77).

The model to predict log length of stay achieved an R² of 0.102 in the derivation set and 0.101 in the validation set. This means that 10% of the variation in the log length of stay was accounted for by the model variables. The most predictive conditions for length of stay were weight loss (0.59 additional days on the log scale, 95% CI 0.49, 0.69), lymphoma (0.38 days on the log scale, 95% CI 0.30, 0.46) and paralysis (0.31 days on the log scale, 95% CI 0.25, 0.37).

Model calibration

Table 5 shows the results of the Hosmer–Lemeshow analysis for the three dichotomous variables across groups, from lowest risk (Group 1) to highest risk (Group 10). Overall, model calibration was good for all three models, with no clear systematic over- or under-prediction of risk in the low, middle, or high-risk groups.

Table 5 Hosmer–Lemeshow test of model calibration for in-hospital mortality (a), increased level of care (b), and 30-day readmission (c)

Discussion

We examined a dataset of more than 50,000 hospitalizations from a large tertiary-care medical center in Israel during a 3-year period prior to the COVID-19 pandemic. We found that the Elixhauser model achieved acceptable prediction of in-hospital mortality, need for escalations of care, 30-day readmissions, and length of stay. These results demonstrate the feasibility of using a standard risk adjustment model, such as Elixhauser, using data from an Israeli hospital.

The IM patient admitted to the hospital has become increasingly more complex over time [23]. Comorbidities are a good predictor of in-hospital mortality [24]. In addition to in-hospital mortality, the Elixhauser approach has been used to predict length of stay, in-hospital adverse events, and is specifically tailored to be used with claims data. The Elixhauser Comorbidity Index is statistically superior to the Charlson Comorbidity Index across disease states, but to our knowledge, has not previously been applied for risk-adjusting IM patients in Israel [25, 26].

In Israel, IM hospital wards are overcrowded with patients of high complexity requiring diagnostic and treatment related services [27]. This puts extra strain on IM wards, which may not be present on more procedure-oriented services, whose reimbursement is more straightforward and more reflective of the actual amount of work involved in caring for such a patient. According to the literature, the DRG payment mechanism is the most common mechanism to pay for internal medicine care in Europe and the United States [28,29,30]. Under the DRG mechanism, patients are grouped by their main admission diagnosis and by severity, such pneumonia without complications, pneumonia with mild complications, and pneumonia with severe complications. These groups receive a fixed amount of reimbursement for the entire hospitalization, which in turn encourages the hospital to minimize length of stay and complications. The classification of diagnoses is based on the International Classification of Diseases (ICD) coding, usually the 10th (ICD-10) or 11th revision (ICD-11). In the DRG-system the ‘treatment nature’ varies by medical specialty. For example, in the Netherlands, there are more than 60 codes for the medical specialty ‘internal medicine’ [28].

In Israel, there are three types of reimbursement for admitted patients: per diem (e.g., medical patients), per procedure (e.g., surgical patients) or fee-for-service [31, 32]. The IM patients can be complex, but it is well-known that there are not enough ward beds in Israel to accommodate all the patients requiring an IM bed—especially in the winter, when respiratory viruses increase the number of hospitalizations to IM services [27]. In addition to these challenges, there is no appropriate system in place to reimburse hospitals according to the differential severity of patients admitted to the IM ward. It is important to emphasize that there may be two separate problems with reimbursement of internal medicine patients in Israel: unfair distribution of funds due to a failure to adjust payment for case complexity, and overall levels of reimbursement that are too low to fully cover the aggregate cost of running internal medicine departments and taking care of this population of paitents. Our contribution is to show that a case mix adjustment model can be made to work in this population. However, the application of such a model to support differential payment would only address the issue of unfair distribution of funds. If the overall level of reimbursement is too low to support hospital activities, a case mix adjustment model cannot address this part of the problem.

In this study, we have shown that the Elixhauser model can be applied and used at SZMC to capture IM patient hospitalization complexity. We established the criterion to support the validity of Elixhauser in our population, based on its ability to predict with acceptable accuracy outcomes including in-hospital mortality, escalation of care, 30-day readmission, and length of stay [33]. A weakness of our study is that it is a single center study; it would be important to repeat this exercise with other Israeli hospitals to ensure that the Elixhauser model works there as well as it did in our population; however, it would be surprising if this will not be the case.

Secondly, while the model was well calibrated across risk strata, the IM patient at SZMC is more complex than most of the other medical centers in Israel. The IM patients at SZMC are older, have more comorbidities, and a higher Charlson score [34]. These factors, combined with a local scarcity of destinations to which patients can be discharged (such as skilled nursing facilities, assisted living, and rehabilitation), lead to longer average lengths of stay at SZMC compared to other Israeli hospitals—although the median length of stay is similar to other institutions. In addition, SZMC has a relatively high percentage of 30-day readmissions compared to other Israeli hospitals [18]—which may reflect the complexity of the patient population as much as anything. Patients treated at SZMC are also older than most patients in Israel, as measured by the median age [18]. However, while SZMC may treat a population that is somewhat unusual in its high complexity, this also provides an ample population to study and apply the Elixhauser model. Lastly, we were unable to confidently identify the main reason why each patient had been admitted to the hospital. The database at SZMC does not distinguish which, if any, of the patient’s diagnosis codes represents the main reason for admission, as is often the case in databases from other locations.

To our knowledge, this is the first instance of applying the Elixhauser model to hospitalizations involving admitted IM patients in Israel. Applying a model like Elixhauser is the first step to accurately capturing the complexity of the admitted IM patient. IM patients require a holistic and individualistic approach to their treatment. Chronic medical illnesses and a long list of medications need to be addressed constantly during their very dynamic admissions, resulting in a need for work that is not well-captured by a per diem (hospital bed-day reimbursement) approach. Each illness that a patient has could be viewed more as a procedure, for which there already exists a system for reimbursement. However, this puts the burden on proper and accurate coding in order to assign the appropriate primary and secondary diagnosis, as well as severity of illness. This approach has been in place outside of Israel for many years. However, this requires properly trained staff, coders, an adequate electronic medical record, and a committed top-down government-based culture change to inpatient IM patient reimbursement. Clearly, coding cannot be perfect, and there are always inaccuracies and even incentives for intentional embellishment of codes, often called "gaming" or “up-coding” [35, 36]. Nevertheless, we would submit that an imperfect system of risk adjustment is still far preferable to none at all, because a total lack of differential payment leads to much worse market distortions than imperfect coding possibly could.

Future studies are needed to model compensation with regard to IM patient complexity in the setting of the present system in Israel, derive a new comorbidity weight on a national level, and continue to validate models such as the Elixhauser model for local use. If done properly, this could come closer to capturing the value provided by all the work performed on IM services, which is currently hard to measure.

Conclusion

In conclusion, we used the Elixhauser model to predict major outcomes among a group of patients hospitalized on the Internal Medicine service at SZMC. Our study is a proof of concept, demonstrating that just like the Elixhauser model has worked in other settings, it can also work here. This finding can be the basis for an effort to more appropriately reimburse IM services in Israel for the work that they are actually doing.