# A physiologically-focused approach to training multi-modality AI algorithms in medicine > **NIH NIH DP2** · UNIVERSITY OF CALIFORNIA, SAN FRANCISCO · 2023 · $1,453,500 ## Abstract PROJECT SUMMARY / ABSTRACT Artificial Intelligence (AI) algorithms have demonstrated success in numerous medical applications. In particular, they excel at analyzing raw medical data formats (such as medical imaging) which has been driven largely by a category of algorithms called deep neural networks (DNN) analyzing single diagnostic tests or modalities indi- vidually. However human disease physiology is rarely confined to one organ system or wholly captured by one diagnostic test. For many of the most clinically-relevant medical applications, the limitation of only being able to analyze one data modality places a major barrier on the types and complexity of medical problems that can be solved. Nearly all important medical decisions require consideration of multiple pieces of information simultane- ously. Current state-of-the-art DNN architectures commonly used in medicine do not readily accept multiple raw data modalities and do not perform highly-complex tasks requiring differential consideration of multi-modal data. To achieve higher-level complex medical reasoning, medical AI algorithms will require fundamental physiologi- cally-focused innovation. The overall objective of this application is to establish a novel physiologically-focused approach to train AI algorithms in medicine that, if successful, will supersede existing AI approaches by func- tioning similarly to the way physician experts triangulate information from multiple sources to arrive at conclu- sions. This research is significant because it addresses the two largest methodologic barriers confronting the real-world application of AI in medicine and that are relevant to all medical specialties: data pre-processing and DNN algorithm architecture. This proposal innovates at the intersection of AI and physiology by developing a new DNN architecture that can accept and learn from multi-modality data in a manner that accommodates a priori medical and physiologic knowledge. Through this novel DNN architecture, resulting algorithms will be able to draw complementary information from multiple inter-related data modalities, similar to how an expert physician considers multiple sources of information to derive a diagnosis or treatment plan. In addition to algorithmic inno- vation, medical AI lacks a scalable approach to perform large-scale data pre-processing, given the heterogeneity of real-world medical data across a wide range of hardware manufacturers and healthcare institutions. Solving this is critical since training data is so important to developing high-performing AI algorithms. This project will also develop an automated approach to perform data pre-processing and harmonization of medical data that is modality-agnostic. To develop and refine these innovations in real-world data, both the data pre-processing pipeline and multi-modal DNN architecture will be applied and prospectively validated to identify heart failure- related phenotypes in a large multi-modality cohort of heart f... ## Key facts - **NIH application ID:** 10687584 - **Project number:** 1DP2HL174046-01 - **Recipient organization:** UNIVERSITY OF CALIFORNIA, SAN FRANCISCO - **Principal Investigator:** Geoffrey H Tison - **Activity code:** DP2 (R01, R21, SBIR, etc.) - **Funding institute:** NIH - **Fiscal year:** 2023 - **Award amount:** $1,453,500 - **Award type:** 1 - **Project period:** 2023-09-06 → 2026-08-31 ## Primary source NIH RePORTER: https://reporter.nih.gov/project-details/10687584 ## Citation > US National Institutes of Health, RePORTER application 10687584, A physiologically-focused approach to training multi-modality AI algorithms in medicine (1DP2HL174046-01). Retrieved via AI Analytics 2026-05-27 from https://api.ai-analytics.org/grant/nih/10687584. Licensed CC0. --- *[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*