Hunt-and-Test Procedures: Design and Derandomization

NSF Award Search · 01002526DB NSF RESEARCH & RELATED ACTIVIT · $175,000 · view on nsf.gov ↗

Abstract

This project addresses a fundamental challenge in testing statistical hypotheses: reliably detecting signals from complex data while avoiding false discoveries due to "double dipping", a practice of unintentionally using the same data to both identify and test hypotheses. Double dipping inflates the rate of false findings, thereby undermining the credibility and replicability of scientific research across various fields, including health, economics, and political science. To overcome this problem, this project develops innovative statistical methods known as "hunt-and-test" procedures, which adaptively seek out meaningful signals in data and rigorously validate them without bias. By introducing novel derandomization techniques, the research also eliminates the randomness and variability inherent in current methods, thereby significantly enhancing the replicability of results. These improved methods will directly support the reliable analysis of scientific data and foster greater public trust in research outcomes. Additionally, the project supports educational advancement by training graduate and undergraduate students in the research project. This project introduces a novel framework for constructing data-adaptive statistical tests through carefully designed and derandomized hunt-and-test procedures. "Hunt and test" randomly splits iid data into A and B, first using A to identify potential signal and then validating the signal with B, thereby maintaining rigorous calibrati

Key facts

NSF award ID
2515385
Awardee
Regents of the University of Michigan - Ann Arbor (MI)
SAM.gov UEI
GNJ7BBP73WE9
PI
Richard Guo
Primary program
01002526DB NSF RESEARCH & RELATED ACTIVIT
All programs
Artificial Intelligence (AI), Machine Learning Theory
Estimated total
$175,000
Funds obligated
$175,000
Transaction type
Standard Grant
Period
09/01/2025 → 08/31/2028