# Statistical Methods for Modern Evidence Syntheses with Multiple Biases

> **NIH NIH R01** · STANFORD UNIVERSITY · 2022 · $334,475

## Abstract

PROJECT SUMMARY/ABSTRACT
 Meta-analyses critically shape clinical recommendations and policy, but their credibility may be
undermined by both within-study biases (e.g., confounding in observational studies) and across-study biases
(e.g., filtering by publication bias). These biases can produce meta-analysis estimates that are too large, too
small, or in the wrong direction. Scientists, clinicians, and health policymakers are increasingly concerned
about these biases given recent empirical evidence that meta-analyses on the same topic can disagree with
one another and with the results of systematic replication studies, which are designed to minimize publication
bias by having independent investigators repeat published studies. This eroding confidence in the published
literature and in meta-analyses represents an epistemic turning point.
 This proposal develops and empirically validates an innovative, domain-independent statistical
framework for quantitatively synthesizing studies subject to within- and across-study biases. Aim 1 will thus
develop novel, domain-independent statistical sensitivity analyses will quantify how results of meta-analysis
estimates might be shifted by within- and across-study biases, that allow bias-corrected synthesis of studies
with these two forms of bias, and that forecast the likely range of results of new studies and the impact of
adding them to an existing meta-analysis. The methods will be made broadly accessible via user-friendly
websites and R software, and their use will be illustrated in meta-analyses on pediatric obesity. Aim 2 will
illustrate the methods' real-world impact and compare their performance to that of existing methods by using
the methods to characterize the credibility of Cochrane database meta-analyses. Aim 3 will involve a
collaboration with “ManyBabies'', an innovative initiative to conduct conceptual replications of landmark results
in developmental psychology. The results of these planned replications will be forecasted using the new
methods of Aim 1 as well as existing methods; the forecasts will be compared studies' results after they are
conducted, providing real performance benchmarks. Aims 2-3 will also provide online “dashboards” allowing
intuitive exploration of the results.
 The immediate-term goal is to develop methods and software that, unlike existing statistical methods,
assess the robustness of a given meta-analysis to the joint effects of within- and across-study biases; that
synthesize and compare results of meta-analyses with those of studies subject to less publication bias (e.g.,
replication studies); and that use potentially biased meta-analyses to plan the optimal design of new studies.
The long-term goal is to calibrate confidence in meta-analyses to more swiftly inform scientifically robust
conclusions that will improve practice and health policy.

## Key facts

- **NIH application ID:** 10479063
- **Project number:** 5R01LM013866-02
- **Recipient organization:** STANFORD UNIVERSITY
- **Principal Investigator:** Maya Mathur
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2022
- **Award amount:** $334,475
- **Award type:** 5
- **Project period:** 2021-09-01 → 2025-08-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10479063

## Citation

> US National Institutes of Health, RePORTER application 10479063, Statistical Methods for Modern Evidence Syntheses with Multiple Biases (5R01LM013866-02). Retrieved via AI Analytics 2026-05-24 from https://api.ai-analytics.org/grant/nih/10479063. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*