INTRODUCTION

Patient experience is valuable because it reflects how patients perceive the care they receive (e.g., communication regarding diagnosis, new medications) within the healthcare system. It is also associated with clinical outcomes, including adherence to medical treatment, patient use of primary care and preventative services, and hospital readmission rates.1,2,3 With the initiation of the Hospital Value-Based Purchasing (HVBP) program, a portion of hospital incentive payments from the Center for Medicare and Medicaid Services (CMS) are based upon patient experience as measured by the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey.4 Press Ganey, an independent survey vendor, is used by almost half of the healthcare systems in the USA to administer HCAHPS surveys. The Press Ganey survey integrates additional patient experience questions and provides real-time data to hospitals to guide quality improvement efforts.5 Hospitals utilizing the Press Ganey survey have higher scores on HCAHPS than hospitals using the HCAHPS survey alone.6

Academic medical centers are unique in that resident physicians at various stages of training provide a significant amount of care to patients under the supervision of an attending physician. Despite this, there is an overall paucity of data evaluating how the addition of residents to hospitalist-led inpatient teams affects patient experience measured by the HCAHPS and Press Ganey surveys. Previous literature evaluating patient satisfaction measured by the Press Ganey survey in an outpatient clinic revealed significantly lower overall satisfaction in all domains when residents were involved in patient care.7

In the inpatient setting, Iannuzzi et al. reported greater patient satisfaction with resident teams compared with advanced practice clinician (APC) (e.g., physician assistant or nurse practitioner) teams in all areas of the HCAHPS physician performance survey domain.8 However, little is known about how resident and APC participation in patient care impacts the HCAHPS and Press Ganey physician performance survey domains in comparison with solo hospitalist teams. Wray et al. showed that patients on solo hospitalist teams had greater satisfaction in their overall care in comparison with those on teaching hospitalist teams, although this was studied with a different survey instrument.9 It is unclear if this correlates to HCAHPS and Press Ganey physician performance domain survey results.

As HVBP incentive payments affect academic and community acute care hospitals, it is imperative to know whether the addition of residents and APCs to hospitalist-led teams impacts survey results. In this study, we sought to compare patient satisfaction scores reported on the HCAHPS and Press Ganey physician performance survey domains between resident, APC, and solo hospitalist teams. We hypothesized that patients admitted to solo hospitalist teams would have greater satisfaction and subsequently higher ratings of physician performance on the HCAHPS and Press Ganey survey instruments.

METHODS

We conducted a retrospective observational cohort study at the University of Utah, a 528-bed academic medical center. An electronic database query was used to identify all patients discharged from the inpatient Internal Medicine service between July 1, 2015, and July 1, 2018. The post-discharge patient experience survey was distributed to all eligible patients by Press Ganey, Inc., and included the HCAHPS 29-question survey instrument as well as an additional 45 questions developed by Press Ganey, Inc. Following HCAHPS survey eligibility criteria, patients were excluded if they were discharged to a skilled nursing facility, hospice, or court/law enforcement, or deceased at the time of discharge; readmitted within 30 days; or had a primary psychiatric MS-DRG diagnosis. Patients were also excluded if they did not complete a post-discharge survey. Baseline patient characteristics collected included: age; gender; self-reported highest educational level, overall health, and emotional health; Charlson comorbidity index (CCI);10 and time of admission. The project was reviewed and deemed exempt by the University of Utah Institutional Review Board (IRB 00104884).

Patients were stratified into one of three cohorts dependent on their assigned treatment team at the time of discharge. The three inpatient team structures were (1) “resident team” comprised of one senior resident (PGY2 or PGY3) and 1–2 medical students, or one senior resident, two interns (PGY1), and 1–2 medical students supervised by a hospitalist physician; (2) “APC team” comprised of 1–2 APCs supervised by a hospitalist physician; and (3) “solo hospitalist” comprised of one hospitalist physician independently managing all patients. During the study period, daytime admissions (6 am–6 pm) were performed by four independently functioning resident teams. Overnight admissions were performed by a dedicated team of two senior residents, two interns, one APC, occasional APC and medical students, and one hospitalist. Patients admitted overnight were redistributed each morning with the majority assigned to an APC team or a solo hospitalist team. The APC and solo hospitalist teams did not accept new admissions during the day.

As the Press Ganey survey has additional questions regarding patient satisfaction with physician performance, we compared patient responses for both the HCAHPS and Press Ganey physician performance survey domains for the corresponding admission. Responses for HCAHPS questions were converted from ordinal (never, sometimes, usually, and always) to binary responses with only a response of “always” receiving credit as only “top box” scores meet the incentive funding requirement by CMS.11 Press Ganey survey responses were evaluated on a Likert scale from 1 to 5 (1-very poor, 2-poor, 3-fair, 4-good, 5-very good) and were not converted to binary responses as there is no pre-defined threshold to our knowledge, analogous to the HCAHPS “top box” score. The datasets generated and analyzed during our study are available from the corresponding author upon reasonable request.

Statistical Analysis

Baseline characteristics were compared using the chi-square test for dichotomous categorical variables, and the t test for continuous variables, where appropriate. The asymptotic Wilcoxon-Mann-Whitney test was used for ordinal categorical variables. Unadjusted survey responses were compared in a pairwise fashion between study cohorts using chi-square for categorical or t test for continuous outcomes. Unadjusted outcomes are reported as frequency and percent or mean with standard deviation. Multivariable regression analysis using generalized linear models with a log-link function and gamma distribution was used for continuous outcomes and multivariable logistic regression was used for binary outcomes.12 Covariates included in regression models were age, gender, CCI, and nighttime admission. Age and nighttime admission were included due to significant differences in baseline values between study cohorts. CCI was included due to a nominal trend of increasing mean CCI between groups, albeit non-significant, to control for confounding effects of chronic illnesses. Gender was included based on an initial analysis that noted significant differences in gender among patients who completed surveys compared with patients without survey results.13 Adjusted continuous outcomes were estimated using marginal effects at the means.12 The p values for pairwise group comparisons were adjusted for multiplicity using the Holm-Sidak multiple comparison procedure.14 A p value cutoff of 0.05 was used to determine statistical significance. Stata/IC version 15.1 (StataCorp, College Station, TX) was used for all analyses.

RESULTS

A total of 12,716 hospital admissions matched our original search criteria. Of these, 5958 were eliminated based on exclusion criteria (Fig. 1), leaving 6759 hospital admissions that met HCAHPS eligibility criteria to receive a survey. A total of 1334 (19.7%) who completed a post-discharge survey were included for analysis, 832 (19.8%) patients discharged by resident teams, 375 (20%) discharged by APC teams, and 127 (18.8%) from solo hospitalist teams. The proportion of completed surveys were not significantly different between teams. Baseline patient characteristics are reported in Table 1. Patients admitted to the resident teams were younger than those admitted to the APC teams (mean age 62.4 years vs. 67.3 years, p < 0.001) and the solo hospitalist team (mean age 62.4 years vs. 68.2 years, p < 0.001). Significantly fewer patients assigned to resident teams were admitted at night compared with those assigned to APC teams (31.4% vs. 49.3%, p < 0.001) and the solo hospitalist team (31.4% vs. 47.2%, p < 0.001), an observation consistent with our institutional admitting process. No other significant differences were observed in baseline characteristics between cohorts.

Fig. 1
figure 1

Consort flow diagram. SNF, skilled nursing facility; MS-DRG, Medicare Severity-Diagnosis Related Group; HCAHPS, Hospital Consumer Assessment of Healthcare Providers and Systems; APC, advanced practice clinician.

Full size image
Table 1 Baseline Characteristics
Full size table

Unadjusted survey responses are reported in Table 2. There was no significant difference in HCAHPS survey responses assessing physician performance between resident, APC, and solo hospitalist teams. All Press Ganey survey responses assessing physician performance were significantly higher for solo hospitalists compared with those for resident teams. Survey responses were similar between APC teams and solo hospitalists, except for the question assessing physician skill, in which solo hospitalists were perceived to have higher physician skill (mean score ± SD, 4.82 ± 0.40 vs. 4.68 ± 0.61; p = 0.040).

Table 2 HCAHPS and Press Ganey Survey Responses by Team, Unadjusted Responses
Full size table

Following multivariable adjustment of survey responses, fewer significant differences were observed. Adjusted survey responses are summarized in Table 3. No significant differences were observed in the adjusted HCAHPS survey responses assessing physician performance between team structures. However, adjusted responses to Press Ganey questions demonstrated significant differences between solo hospitalist and resident teams, with solo hospitalists having higher scores in three areas: time physician spent with you (4.58 vs. 4.38, p = 0.050); physician kept you informed (4.63 vs. 4.43, p = 0.047); and physician skill (4.80 vs. 4.63, p = 0.027). Similar to unadjusted responses, solo hospitalists were reported to have higher physician skill compared with APC teams (4.80 vs. 4.69, p = 0.042) after controlling for differences in baseline characteristics and adjusting p values for multiple comparisons.

Table 3 HCAHPS and Press Ganey Survey Responses by Team, Adjusted Responses
Full size table

DISCUSSION

To our knowledge, this is the first study comparing patient satisfaction in physician performance on inpatient resident, APC, and solo hospitalist teams as measured by the HCAHPS and Press Ganey survey instruments. The adjusted Press Ganey survey results revealed no significant differences in physician performance on resident and APC teams. However, the adjusted Press Ganey survey results suggest that physicians on solo hospitalist teams were perceived as more skilled than physicians on resident and APC teams. In addition, patients reported that physicians on solo hospitalist teams spent more time with them and did a better job of keeping them informed compared with physicians on the resident team. These differences were not observed in the HCAHPS physician performance survey responses, therefore having no impact on HVBP incentive payments with regard to physician performance.

Previous work by Iannuzzi et al. observed greater patient satisfaction with resident teams in comparison with APC teams on the HCAHPS survey.8 However, our study found no differences between resident and APC teams on either the HCAHPS or the Press Ganey surveys. It is possible that these survey instruments are not sufficiently sensitive to detect small differences in patient satisfaction, or our study was underpowered to detect these small differences. We believe our work is novel in that it also evaluated patient satisfaction on solo hospitalist teams in addition to resident and APC teams. As HVBP incentive payments affect academic and community acute care hospitals, it is important to note that the addition of resident or APC team members on hospitalist-led teams had no significant impact on HCAHPS physician performance domain results.

While our HCAHPS survey results were similar for all three team structures, differences were observed in the Press Ganey results across team structures. The reasons for these differences are uncertain, but it is important to highlight subtle differences in the wording of HCAHPS and Press Ganey responses designed to evaluate physician performance that may influence survey responses. HCAHPS questions measure how often a service was provided using the scale of never, sometimes, usually, or always. Press Ganey questions measure how well a service was provided using the scale of very poor, poor, fair, good, and very good.6 Our HCAHPS survey results suggest that resident, APC, and solo hospitalist teams perform similarly in the consistency of care. In contrast, our Press Ganey survey results suggest patients perceive a solo hospitalist team provides higher quality of care compared with resident teams in three areas: physician skill, time physician spent with you, and physician kept you informed. Also, patients perceive that solo hospitalists are more skilled than physicians on APC teams.

It seems intuitive that patients would view solo hospitalists as more skilled in comparison with resident and APC teams given that patients have direct interactions with only an attending hospitalist. However, it is interesting that patients also perceive that solo hospitalists spend more time with them and do a better job keeping them informed than physicians on resident teams. If patients knew who their attending hospitalist was, perhaps this would make sense, as often on resident teams the intern and resident spend a greater amount of time than the hospitalist in the face-to-face counseling and coordination of care.15 Yet, we know patients cannot identify their physician the majority of the time and often do not understand the roles and responsibilities of each team member.9, 15,16,17 It is plausible that when hospitalist-led teams have more than one provider, physician team members may not clearly articulate their role to the patient, potentially leading to mixed messages, and the perception of an inconsistent plan.18, 19 While these differences in patient satisfaction do not affect incentive payments, they support prior conclusions that patients have a better experience on solo hospitalist teams, which is important given its tie to clinical outcomes.3,4,5, 9 The majority of the HCAHPS and Press Ganey physician performance survey domain questions focus on the ability of the provider to communicate effectively; however, many resident physicians do not receive formal patient-provider communication training.20, 21 Communication is a skillset that can be learned with practice, and as we continue to rely on patient experience as a marker of healthcare quality, these results should motivate us to think about how we train our future physicians.22

The Press Ganey survey is utilized in conjunction with the HCAHPS survey by almost half of U.S. hospitals because results are available in a timelier manner and allow health systems to evaluate and respond to feedback sooner.5 One limitation of both survey instruments is the inability to directly attribute survey responses to an individual provider, especially when multiple providers of varying levels of training and/or different specialties are involved in the care of a single patient. There are times when an APC, resident, or intern may be perceived as the primary physician by the patient based upon the amount of time these team members spend in the counseling and coordination of care. Ultimately, the survey is attributed to the attending physician at the time of discharge. While our results show that this does not impact HCAHPS results, it may be worth considering the addition of physician performance questions focused on patient satisfaction with residents and APCs in addition to attending hospitalists on the Press Ganey survey. When this was adopted for evaluation of emergency room residents, the additional questions did not affect response rates, and the survey data was provided to the residents as part of structured review sessions which allowed them to act on the feedback promptly.23

Several limitations of this study should be acknowledged. First, this is a single-center study which may limit its generalizability to other practice settings. Second, our survey response rate was 19.7%, which is lower than the national average of 26%.11 Interestingly, the national HCAHPS response rate has significantly fallen from 33% in 2008 to 26% in 2018 with some experts attributing low response rates to the lack of a digital survey mode and length of the survey.11, 24, 25 The precise reasons for our low response rate are unknown. Third, our study cohorts were not evenly distributed with the majority of patients assigned to resident teams and the fewest patients assigned to a solo hospitalist team. This imbalance may have limited the power to detect a significant difference among HCAHPS responses. Last, we were unable to control for other factors that impact overall patient experience (e.g., environmental factors, diagnosis/treatment) that may not directly reflect the care delivered by the physician.

CONCLUSION

The Press Ganey survey results suggest that patients have greater satisfaction with solo hospitalist teams in three domains: physician skill, time physician spent with you, and physician kept you informed. However, these differences were not observed on the HCAHPS physician performance survey domain, suggesting team structure should not impact HVBP incentive payments. Further refinements to patient satisfaction survey instruments may be necessary to improve response rates and accurate attribution to healthcare providers.