154 - Development of a Second-Generation High Value Care Rounding Performance Tool

Publication Number: 154.4455

William A. Frese, University of Illinois College of Medicine, Peoria, IL, United States; Yanzhi Wang, University of Illinois College of Medicine, Peoria, IL, United States; Wei-Cheng Hsiao, University of Illinois College of Medicine Peoria, Peoria, IL, United States; Keith Hanson, University of Illinois College of Medicine at Peoria, Peoria, IL, United States

Background: Given inpatient hospitalization’s large contribution to national healthcare costs, providers should engage patients and families in high-value care (HVC) conversations as part of family centered rounding (FCR). The HVC Rounding Tool (HVC RT; McDaniel, 2018), comprised of ten measures assessing clinical practices in quality, cost, and patient valuation domains, is the first and only published tool that evaluates providers’ HVC performance. However, the HVC RT’s original design has limitations, including no previous psychometric validation of its structure, and its relative length.

Objective: The purpose of this study is to establish the HVC RT’s psychometric properties and to explore creation of a shorter, validated ‘second-generation’ HVC performance assessment tool.

Design/Methods: This study leveraged a series of psychometric validation and measure reduction tests on a tertiary center’s pediatric hospitalist teaching service’s HVC FCR performance data that used the HVC RT. A Kuder Richardson (KR-20) test first was applied to explore the HVC RT’s measures’ internal reliability to respective domains, with unacceptable measures being excluded. Correlation and exploratory factor analysis (EFA) was then performed on the remaining, validated measures to explore domain construct and further measure reduction.

Results: A total of 365 FCR encounters were analyzed, exceeding the minimum threshold (n=300) for analysis. None of the KR-20 values in HVC RT’s domains were within preferable reliability range (≥0.7), with one domain particularly requiring removal of a measure to achieve minimum reliability (>0.5). EFA further reduced the HVC RT’s original construct by one-third, recategorizing its original three-domain, ten-measure construct into a two-domain, seven measure construct, comprised of a ‘cost-effective care’ domain with four measures, and a ‘shared-decision-making’ domain with three measures. Kaiser-Meyer-Olkin measure of sampling adequacy, root mean square residual, retained measure factor-loading, and measure communality values, which analyze model strength from various perspectives, were all respectively strong at 0.81 (≥0.7 preferred), 0.054 (≤0.05 excellent), ≥0.65 (>0.6 strong), and ≥0.46 (≥0.4 acceptable).

Conclusion(s): The original HVC RT has suboptimal psychometric validity, requiring at least one measure’s elimination, and suggesting further recategorization of measures and domains for better validity. Our EFA model, with its strong supporting data, accomplishes these goals by suggesting a shorter, second-generation HVC Performance Rounding Tool, which may be easier to implement.

TABLE 1. HVC Rounding Tool’s Measure Performance Frequency, Domain Kuder-Richarson (KR-20), Factor Loading, and Communality Values (n=365)
Table condenses several key analyses performed on data collected using the HVC Rounding Tool: In addition to providing measure description and performance frequencies for background, the KR-20 analyzes measures' internal consistency/reliability within their respective domains, indicating measure #1's elimination in order for the Quality domain to achieve minimum reliability. After the additional elimination of measure #4 (see TABLE 2), a scree plot and proportion criterion (data not shown---will be presented in poster) showed a two-factor solution to be best (Kaiser-Meyer-Olkin measure of sampling adequacy and root mean square residual calculations supported 2-factor model to be strong), with subsequent factor loading values presented, indicating measure #3's elimination. There was no cross-over of measures onto factors. Finally, communality represents the variance explained by each measure within its EFA modelling, with 57% of measures very near or above an ideal threshold of =/>0.6 .

TABLE 2. Polychoric Correlation Coefficients for Validated HVC Rounding Tool Measures (n=365)
As part of determining suitability of applying EFA to data, a correlation procedure is performed as a prerequisite (in this case, a polychoric correlation given binary nature of data). Measures should generally have a high correlation value (i.e. =/>0.3), but not too high of a correlation (i.e. =/>0.8) that suggests collinearity and weakens the EFA model. For circumstances of highly-correlated measure-pairs, one measure should be excluded from EFA modelling. Measures #4 and #10 were found to highly correlated. In determining which measure to retain/exclude from EFA, separate EFA modelling for each measure, excluding the other, was performed. Kaiser-Meyer-Olkin measure of sampling adequacy and root mean square residual calculations (i.e, measures of EFA model strength) were stronger for measure #10, excluding measure #4. Based on this comparative analysis, measure #10 was retained and measure #4 was excluded from further EFA modelling (i.e. factor loading and communality calculations presented in TABLE 1).

TABLE 3. Second-Generation HVC Rounding Performance Tool (HVC RT2)
Presents a psychometrically-validated Second Generation HVC Rounding Performance Tool, which if applied to this study's data (n=365) compared to the original HVC Rounding tool, would translate to 1,095 less measures auditors would need to record.