# Integrative Deep Learning Models for Multimodal Markers of Cancer Treatment Outcomes

> **NIH NIH F99** · CASE WESTERN RESERVE UNIVERSITY · 2024 · $39,869

## Abstract

PROJECT SUMMARY
The most pressing challenge in oncology is the need for accurate biomarker-driven prognostic/predictive risk-
stratification to identify patients who are unlikely to benefit from standard of care (SOC) chemotherapy early in
their treatment, as they might be better candidates for alternative therapies (e.g., genome-targeted agents,
immunotherapy). Unfortunately, only ~31% of eligible cancer patients achieve partial/complete response to
cytotoxic chemotherapy. For instance, over 40% of GB patients will inevitably recur within 6-8 months after
chemotherapy, suggesting that they could have been better candidates for newer experimental therapies. A
significant challenge in management of these patients is thus, segregating GB patients based on their
outcomes/response to treatment. Similarly, the aggressive chemoradiation protocol for rectal cancers results in
up to 70% of patients achieving 3-year disease-free survival. However, reliably determining which rectal cancer
patients will not benefit from this protocol could allow for targeted adjuvant therapy to ensure optimal outcomes.
Considering a “micro” to “macro” view of the tumor, comprehensive clinical evaluation for cancer involves
acquiring multi-scale data, including radiology (e.g., CT, MRI) which provides macroscopic morphology and
structural tumor details, histology images containing rich phenotypic information at cellular level, molecular data
(e.g., genome sequencing, gene expression, epigenomics, also known as multi-omics) which captures the
underlying biological processes, and the clinical data (e.g., age, sex). Ability to comprehensively combine
disparate sources of information through computational approaches could enable discovery of new prognostic
and predictive markers to reliably assess risks associated with response of chemotherapy and clinical outcomes.
The F99 phase of this proposal continues my dissertation research on developing deep leaning (DL) multimodal
models (mmSurvNet) to build prognostic markers for clinical outcomes, by combining MRI and digital pathology,
in rectal and GB tumors. My research for the F99 phase is driven by the hypothesis that DL models, using co-
registered pathology and radiology images that capture spatially co-localized tumor biology, can yield robust and
reliable prognostic integrated-markers to predict clinical outcomes. Towards this, I will construct multimodal
survival (mmSurvNet) models employing DL architectures that maximize spatial information across pathology
and radiology. The attention maps for mmSurvNet will allow for establishing biological relevance, by spatially
correlating radiology images with corresponding pathology which will contain annotations of known prognostic
tissue characteristics. My proposed K00 phase will involve building predictive DL models (mmPredictNet)
through incorporation of genomic, clinical, longitudinal data together with radiology and pathology images to build
integrated markers predictive of re...

## Key facts

- **NIH application ID:** 10988982
- **Project number:** 1F99CA294169-01
- **Recipient organization:** CASE WESTERN RESERVE UNIVERSITY
- **Principal Investigator:** Olivia Krebs
- **Activity code:** F99 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2024
- **Award amount:** $39,869
- **Award type:** 1
- **Project period:** 2024-09-01 → 2026-08-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10988982

## Citation

> US National Institutes of Health, RePORTER application 10988982, Integrative Deep Learning Models for Multimodal Markers of Cancer Treatment Outcomes (1F99CA294169-01). Retrieved via AI Analytics 2026-05-25 from https://api.ai-analytics.org/grant/nih/10988982. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*