Project Summary / Abstract Colorectal cancer (CRC) is the third most commonly diagnosed malignancy and the second leading cause of cancer death worldwide. There is an unmet need for accurate, cost-efficient, and broadly accessible risk- stratification tools to identify patients at increased risk of recurrence , who are most likely to benefit from adjuvant therapy. Current standard-of-care risk stratification approaches are inadequate. Every CRC surgical candidate undergoes pathologic and radiologic evaluation of their tumor; these two modalities represent a rich, readily accessible and, thus far, underutilized resource for developing new risk-stratification tools. Deep learning (DL) has demonstrated great potential for augmenting physicians on an increasing range of diagnostic and prognostic tasks in pathology, radiology, and clinical medicine. We hypothesize that applying integrated DL-based analysis to multimodal (pathologic, radiologic, and electronic medical record (EMR)) data will yield greatly improved stratification of CRC patients for adjuvant treatment planning. We propose to build the first comprehensive, publicly-available, expert-annotated multimodal CRC dataset for deep learning, containing annotated CRC pathology whole-slide images (WSI), preoperative CT and MRI images, and structured clinical EMR data. Using this dataset, we will develop both single and multi-modality DL models for risk stratification of surgically-resectable (Stage I-III) CRC patients.To test our hypothesis, we will compare the performance of multi-modality models with that of single-modality models and existing methods of stratification. This project benefits from the complementary expertise and resources of a unique interdisciplinary team spanning the fields of machine learning, pathology, radiology, and oncology.