Project Summary Agreement studies, used in the evaluation of newly-developed biomarkers and diagnostic tests, depend upon statistical methods proposed nearly 50 years ago. As Alzheimer’s disease has entered into an era where neuroimaging biomarkers are incorporated into the diagnostic strategy, the progression of our statistical methods must match that of our medical growth. These statistical methods contain significant flaws, such as depending on sample disease prevalence, having complex interpretations, imposing restrictive experimental designs and failing to account for risk factors. The current proposal seeks to advance the statistical methods used in the development of biomarkers and diagnostics, and, in doing so, will identify characteristics important for diagnosing and predicting Alzheimer’s disease status. We hypothesize that our novel statistical contributions will correct these flaws in our approach to biomarker and diagnostic test development. It will allow for a more informative, interpretable and robust way to quantify the agreement and accuracy of medical tests. For our first aim, we will develop a novel statistical methodology to quantify agreement, centering our framework on mixed effect models. We will compare our approach with traditional methods among providers evaluating neuroimaging biomarkers. We will leverage data from the NIA- funded R01 Alzheimer’s Prevention through Exercise (APEx) study and enhance it with primary data collection of neuroimaging interpretations by a diverse sample of providers. For our second aim, we will determine how the sampling design of contemporary agreement studies influence predictive accuracy. Simulation “in-silico” studies will be performed, reproducing many scenarios present in the literature. Additionally, data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) will be leveraged and measures informing accuracy, such as sensitivity and specificity, will be calculated. We will do so under traditional approaches and our novel statistical framework, demonstrating that current methods are merely a special case of our more robust model. This will highlight limitations of current approaches. The broad, long term objectives of this proposal are two-fold. First, we aim to develop a robust and generalized statistical method for evaluating agreement and accuracy of biomarkers and diagnostic tests. By using mixed effects models and corresponding sampling designs, we will overcome flaws present in traditional approaches and gain advantages, such as easily interpreted measures and generalizable results. The second objective is to facilitate a rigorous training plan that will provide a foundation for my future career as a physician-statistician who focuses on biomarker methodology. While the combination of these objectives aligns with the purpose of my current MD-PhD program and this NIH Pre-doctoral Training F30 award, the statistical advancements are applicable to any medical condition, and ...