Benchmarking Hospital Quality: Template Matching versus Conventional Regression Approaches

NIH RePORTER · VA · I01 · · view on reporter.nih.gov ↗

Abstract

Background: Identifying and remediating low-quality care is at the heart of systematic quality improvement, particularly in VA. However, cross-hospital comparisons to identify under-performing hospitals are limited by differences in patient case-mix and illness severity. Clinicians consider the current approach to benchmarking with conventional regression to be unfair, unclear, and unhelpful. While substantial resources are devoted to hospital benchmarking, the current return on this investment is limited because clinicians do not understand or trust the methods. The National Academy of Medicine has recently called for investing into the science of performance measurement and for increasing its transparency and validity. Template matching has been proposed as an alternative methodological approach to benchmarking that is fair, clear, and helpful. However, this new approach has never been tested outside of limited research settings. Specific Aims: To test the utility of template matching for comparing quality of care across VA's diverse acute care hospitals, this project will assess the feasibility, accuracy, and interpretability of this approach. Specifically, the project will: (A1) Feasibility: Develop and optimize two template matching approaches for comparing 30-day mortality across VA hospitals. (A2) Accuracy: Compare the ability of template matching versus conventional regression to correctly identify under-performing hospitals. (A3) Interpretability: Compare the interpretability and credibility of hospital performance data generated from template matching versus conventional regression models with clinical leaders. Anticipated Impact: The overarching goal of this proposal is to improve the care of hospitalized Veterans. This proposal will apply promising, new template matching approaches for benchmarking to the diverse VA healthcare system and complete a multi-faceted evaluation of these approaches. We will consider not only the feasibility and methodological rigor of template matching, but also the interpretability, credibility and accountability of the data. We expect that template matching will be feasible for benchmarking VA acute care hospitals, that it will identify under-performing hospitals at least as well as current benchmarking with conventional regression, and that it will be more interpretable and credible to VA Chiefs of Medicine than current performance reports. We anticipate many future expansions to this work, including the use of template matching for VA to private sector comparisons. Unique Features and Innovation: This proposal is innovative because it will simultaneously (1) advance the statistical state-of-the-art in hospital benchmarking, (2) develop the necessary infrastructure in consultation with operational partners to use template matching in VA, and (3) evaluate this new approach not just statistically, but also with the actual users it is supposed to inform. The results are highly relevant to VA, whose performa...

Key facts

NIH application ID
10308540
Project number
5I01HX002546-04
Recipient
VETERANS HEALTH ADMINISTRATION
Principal Investigator
Hallie Christine Prescott
Activity code
I01
Funding institute
VA
Fiscal year
2021
Award amount
Award type
5
Project period
2018-03-01 → 2022-02-28