# Scaling Volumetric Imaging, Analysis and Science Communication Using Immersive Virtual Reality

> **NIH NIH R44** · ISTOVISR · 2024 · $1,205,971

## Abstract

Over the past 15 years, new microscope technologies and methods for high throughput imaging
have revolutionized structural biology by extending the resolution and scale of datasets in 3
dimensions. The resulting image volumes are more typically hundreds of GB to even tens of TB
and for large volume electron microscope images of brain, can approach PB sizes. These file
sizes pose challenges for image analysis, and communication of a representative set of raw
data and quantification. Large files contain many structures, and require machine learning (ML)
strategies in a context that permits error correction. Scientific communication requires tools for
ready access to raw data, and more efficient methods to communicate the rapidly accumulating
sets of scientific information. The rapidly accumulating digital library also affords a resource for
teaching and training, which is largely untapped.
We propose to leverage virtual reality (VR) to transform each of these challenges, capitalizing
on natural abilities for stereoscopic vision and pattern recognition and, for scientific
communication, teaching and training, auditory processing to process language and localize
sounds. Based upon the tool base and direct volume rendering of large files that we have
established in our VR software, called syGlass, we will first expand modern domain learning and
so-called meta-learning techniques in the ML field to analyze images with few iterations from
object counting to object tracking and tracing (Aim 1). Next, we will capitalize on new
technologies for cloud rendering to significantly mitigate the hardware costs for adoption of
syGlass (Aim 2). Finally, we will provide novel tools to efficiently generate narrated scientific
presentations in VR for use in the lab setting, as manuscript publications, and for production of
educational materials (Aim 3). The complexity of the brain offers a challenging testbed for
teaching and training. In each of these Aims, we will introduce paradigm shifts in the analysis of
the large data volumes, and communication of 3D and 4D data to colleagues and non-experts.

## Key facts

- **NIH application ID:** 10773628
- **Project number:** 5R44MH125238-04
- **Recipient organization:** ISTOVISR
- **Principal Investigator:** Gianfranco Doretto
- **Activity code:** R44 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2024
- **Award amount:** $1,205,971
- **Award type:** 5
- **Project period:** 2020-05-01 → 2026-01-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10773628

## Citation

> US National Institutes of Health, RePORTER application 10773628, Scaling Volumetric Imaging, Analysis and Science Communication Using Immersive Virtual Reality (5R44MH125238-04). Retrieved via AI Analytics 2026-06-11 from https://api.ai-analytics.org/grant/nih/10773628. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*