Evaluation Core (EC)

NIH RePORTER · NIH · U54 · $1,032,928 · view on reporter.nih.gov ↗

Abstract

PROJECT SUMMARY/ABSTRACT The Evaluation Core (EC) proposal outlines its contributions to the overall aims of the Coordination and Evaluation Center at UCLA (CEC), as part of the overall NIH Diversity Program Consortium (DPC) and its goal of enhancing the diversity of the NIH-funded workforce. In collaboration with the other CEC cores, and DPC grantees, the EC is responsible for the consortium-level evaluation of the short, medium, and long-term outcomes of BUILD grantee programs across student, faculty, and institutional levels, and of previously funded NRMN components on medium and long-term outcomes. The EC continues to update DPC-approved Hallmarks of Success and refine the Consortium-wide evaluation plan to guide our work. Our multi-method evaluation builds on multiple case studies, cross-site analysis, a robust quasi-experimental design using multilevel statistical analysis, and trend analyses of key quantitative indicators. For individual-level data on BUILD participants, we integrate information from the Higher Education Research Institute (HERI) surveys of freshman and senior students and of faculty nationally, CEC annual surveys of BUILD sites, event-level “tracker” data about program participation, institutional record (IR) and national IPEDS data. We use this rich set of data to compare the achievement of Hallmarks for BUILD participants with nonparticipants at the same institution, across BUILD institutions, and between BUILD and non-BUILD institutions. The longitudinal dataset and extensive covariates improve our ability to attribute causality to the interventions. National institutional level data covering multiple years from IPEDS supports analysis of potential institutional-level changes by comparing trends at BUILD versus similar non-BUILD sites. For NRMN, we draw on baseline data collected by NRMN from participants and supplement it with CEC annual surveys to assess the long-term outcomes of Hallmarks, comparing the statistically adjusted achievements of Hallmarks by specific program participants versus those in the NRMN database who did not participate in the particular program. Innovative aspects of our evaluation design include: (i) access and flexibility to build on established metrics in HERI national surveys of undergraduate education, (ii) use of social network analysis to better characterize the outcomes of career development strategies, and (iii) the analysis of BUILD programs across multiple institutions to provide the statistical power to disaggregate the outcomes of specific intervention components and identify differential effects on subpopulations, including program participants from underrepresented groups. Jointly with the DCC, the EC will support any needed updates to data collection tools and processes, Hallmarks, and analytic approaches to ensure the comprehensive assessment of the impact of BUILD (Phase 1 & 2) and NRMN (Phase 1) activities. The EC will also contribute to the CEC's development and disseminatio...

Key facts

NIH application ID
10022513
Project number
5U54GM119024-07
Recipient
UNIVERSITY OF CALIFORNIA LOS ANGELES
Principal Investigator
Steven Paul Wallace
Activity code
U54
Funding institute
NIH
Fiscal year
2020
Award amount
$1,032,928
Award type
5
Project period
— → —