PROJECT SUMMARY – Technology Research and Development Project #3 There are unmet needs in critical clinical care scenarios (e.g., surgery and intensive care) namely the lack of real-time intraprocedural imaging and pathologic data, intelligent systems for visualization, and integration this multimodality data with other clinical data for real-time decision guidance. TRD3 will develop deep learning (DL), machine learning (ML), artificial intelligence (AI), and visualization (VIS) tools to address these challenges. To accomplish this objective the research team will undertake four Specific Aims: In Aim 1, the research team will build data-driven instruments by jointly optimizing the optical hardware and the back-end machine learning model for a given task. Optimized iFLIM and iDOS instruments with increased capabilities and higher SNR will be developed for clinical use. In Aim 2, the research team will develop effective and expressive visualization interfaces and human comprehension of multimodality imaging data. These tools will provide critical information for critical decision making and improve clinical workflow. In Aim 3, the research team will develop new AI tools to integrate heterogenous multimodality data to predict patient outcome. The multi-model data integration approach will overcome the limitations of each single modality being considered in isolation. Finally, in Aim 4, the research team will incorporate these new techniques into clinical workflow to provide real-time feedback for surgical guidance. By accomplishing these aims the research team will develop and validate a set of advanced analytical methods with AI/ML/DL for intelligent instrument design, data/image analysis, visualization, and clinical decision making. Strong interactions and shared resources between this TRD and TRDs1 and 2 will enable performance advancements in the imaging and inference capabilities. The combination of these approaches will pave the way for choosing highly personalized treatments based on predictions of individual patient outcome.