# Measuring the Multilevel Outcomes of Children's Mental Health Services Implementation

> **NIH NIH R21** · UNIVERSITY OF WASHINGTON · 2020 · $233,250

## Abstract

PROJECT SUMMARY
Implementation science aims to close the science-to-practice gap, and progress has been made toward
consensus on implementation strategies and outcomes. Implementation studies are primarily focused on direct
outcomes of implementation strategies, despite evidence that implementation efforts in complex settings such
as children’s mental health services lead to a host of multilevel outcomes. Multilevel outcomes include
constructs that are not the primary focus; they are indirect, unplanned, or more salient to other stakeholders
than the researcher. A systematic review of children’s mental health implementation strategies found only one
study systematically investigated unplanned outcomes. The long-term goal of this study is to improve
implementation science in children’s mental health services by developing a taxonomy and decision tool for
incorporating multilevel outcome measures, and develop one pragmatic measure as relevant to consumers.
Including multilevel outcomes may identify important and unexpected results to inform quality improvement,
usability, and the effectiveness of interventions as well as learning about negative downstream. Many
implementation research projects lack the awareness of or resources to measure multilevel outcomes, which
we believe is caused by the current absence of existing guidelines for selecting the most appropriate multilevel
outcomes for common implementation strategies. Through use of theory, expert consensus, and stakeholder
participation, we will identify a taxonomy of multilevel outcomes that are linked to implementation strategies
and are most relevant to four stakeholder roles (consumers/families, providers, policy makers, and
researchers). This will be done through three rounds of surveys with a pool of expert participants representing
each of these roles, who will rate implementation strategies and associated outcomes as relevant to their
stakeholder role. Results will be used to create a decision making tool to identify and measure the full balance
of outcomes of implementation strategies in children’s mental health services, which would have several
benefits. It would provide a more complete and democratic picture of the cascading and occasionally surprising
impacts of implementation efforts, and contribute to theory modification. It would help implementers and policy
makers increase the usability and effectiveness of interventions, make better cost-benefit choices, provide
opportunities for the prevention of undesirable outcomes, enable innovators to highlight positive multilevel
outcomes, and improve precision in predicting implementation.

## Key facts

- **NIH application ID:** 9892909
- **Project number:** 1R21MH119360-01A1
- **Recipient organization:** UNIVERSITY OF WASHINGTON
- **Principal Investigator:** Michael David Pullmann
- **Activity code:** R21 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2020
- **Award amount:** $233,250
- **Award type:** 1
- **Project period:** 2020-02-05 → 2021-12-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/9892909

## Citation

> US National Institutes of Health, RePORTER application 9892909, Measuring the Multilevel Outcomes of Children's Mental Health Services Implementation (1R21MH119360-01A1). Retrieved via AI Analytics 2026-05-23 from https://api.ai-analytics.org/grant/nih/9892909. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*