# A holistic approach to interactive medical diagnosis via large language model

> **NIH NIH P20** · UNIVERSITY OF IDAHO · 2024 · $153,002

## Abstract

Melanoma is the primary cause of most skin cancer-related fatalities. Timely detection and appropriate 
treatment considerably boost the survival chances of melanoma patients. Dermoscopy image-based 
melanoma diagnosis, noted for its precision and ease, is eagerly anticipated. However, despite AI-based 
diagnostic systems achieving accuracy levels comparable to human dermatologists, their broader 
application is hindered by concerns regarding privacy, legality, transparency, and interpretability. 
Addressing these issues to enhance the intelligence and reliability of smart diagnostic systems, 
particularly in offering interactive features, is imperative. 
 
The advent of large language models (LLMs), pre-trained on extensive textual data, presents potential 
solutions to these challenges. This research aims to establish an AI-assisted diagnosis system for 
dermoscopy image-based melanoma, combining high recognition accuracy with the interactive 
capabilities of LLMs. Such a system could aid dermatologists in diagnosis and act as virtual patients for 
medical training, creating a novel framework for interactive AI-assisted diagnostics using LLMs and 
marking a pioneering step in applying LLMs to dermoscopy image-based melanoma diagnosis. 
Aim 1 focuses on developing an AI-driven pipeline to automatically generate textual case descriptions 
from medical images. Aim 2 involves creating LLMs specialized in melanoma dermoscopy diagnosis. Aim 
3 aims to develop and validate AI-driven diagnostic assistants and virtual patients in medical educational 
settings. This research could significantly impact computer-aided skin cancer diagnosis by bridging the 
gap between medical image analysis and LLMs. Furthermore, it will pioneer integrating AI-driven virtual 
patients into medical education, enhancing the training of future medical professionals.

## Key facts

- **NIH application ID:** 11163592
- **Project number:** 5P20GM104420-10
- **Recipient organization:** UNIVERSITY OF IDAHO
- **Principal Investigator:** BOYU ZHANG
- **Activity code:** P20 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2024
- **Award amount:** $153,002
- **Award type:** 5
- **Project period:** 2015-03-15 → 2026-06-30

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/11163592

## Citation

> US National Institutes of Health, RePORTER application 11163592, A holistic approach to interactive medical diagnosis via large language model (5P20GM104420-10). Retrieved via AI Analytics 2026-05-24 from https://api.ai-analytics.org/grant/nih/11163592. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*