# Deep Learning for Individualized Treatment Effect Inference with Multimodal Depiction

> **NIH NIH R21** · YALE UNIVERSITY · 2024 · $234,375

## Abstract

PROJECT SUMMARY
Predicting the outcome of a treatment given the pre-operative patient status, i.e., individualized treatment effect
(ITE) inference, is of great clinical importance for precise treatment planning. For example, the ITE w.r.t. survival
time estimation of glioblastoma (GBM) patients undergoing different treatments enables assessing these possible
multi-treatments by answering the question: ”would this patient have lived longer (and by how much), had an
alternative treatment been applied?” Improving beyond subjective experience-driven therapy, the widespread
accumulation of big medical data offers unprecedented opportunities for the data-driven deep learning (DL)
algorithms to learn the underlying causal relations between multimodal depict patient imaging and clinical data,
multi-treatments, and corresponding ITE. The practical ITE prediction requires a DL framework, which is largely
unavailable at present. The current methods have limitations, including only utilizing the partial and incomplete
status depiction, not applicable to multi-treatment on the outcome, and neglecting the ordinal ITE labels. In addition,
the lack of reliability information and interpretability in the conventional DL model also hinders its large-scale
clinical implementation. We propose to use our previous successful DL model to take both multimodal status and
multi-treatments for accurate ITE inference for both factual and counterfactual cases with either continuous or
ordinal labels. We will further establish a deep self-training scheme for reliable and interpretable ITE inference with
quantified uncertainty and visualized DL-focused pathology region. The overall goal of this project is to develop an
accurate, reliable, and interpretable pipeline for ITE inference by leveraging our advanced DL technique, which
can be widely generalizable. This concept could significantly advance individualized treatment planning. The
overall hypothesis is that the proposed solution can offer a unique opportunity to characterize causal relations
among multimodal status depictions, multi-treatments, and corresponding ITEs with the novel DL model, which
is not provided by current direct models. In addition, enabling the reliability quantification and interpretation of
the underlying patterns of DL decisions could open a new window for ITE outcome utilization and treatment-
specific pathology patterns investigation, thus leading to a multitude of new applications. The specific aims of this
exploratory proposal are (1) to develop a multimodal multi-treatment DL framework for accurate ITE inference, (2)
to establish a deep self-training scheme for reliable and interpretable ITE inference with calibrated uncertainty and
visualized 4D (3D+modal) gradient activation. We will apply the proposed DL-based ITE inference framework to
the clinical GBM survival dataset with different resections and test it based on various figure-of-merits. Successful
completion of the project will provide ...

## Key facts

- **NIH application ID:** 10886295
- **Project number:** 1R21EB034911-01A1
- **Recipient organization:** YALE UNIVERSITY
- **Principal Investigator:** Xiaofeng Liu
- **Activity code:** R21 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2024
- **Award amount:** $234,375
- **Award type:** 1
- **Project period:** 2024-06-01 → 2027-03-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10886295

## Citation

> US National Institutes of Health, RePORTER application 10886295, Deep Learning for Individualized Treatment Effect Inference with Multimodal Depiction (1R21EB034911-01A1). Retrieved via AI Analytics 2026-05-25 from https://api.ai-analytics.org/grant/nih/10886295. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*