# Streamlining Volumetric Imaging, Analysis and Publication Using Immersive Virtual Reality

> **NIH NIH R44** · ISTOVISR · 2021 · $499,966

## Abstract

Over the past 15 years, new imaging technologies and methods for high throughput imaging
have revolutionized structural biology by extending the resolution and scale of collected images
in 3 dimensions. The resulting image volumes are more typically hundreds of GB to even tens of
TB and in some cases approach PB sizes. These file sizes pose challenges for image
acquisition, image analysis, and communication of a representative set of raw data and
quantification. Image acquisition runs can be lengthy and expensive, and often errors are not
identified until after the completion of scanning. Large files contain many structures, and require
machine learning (ML) strategies in a context that permits error correction. Scientific
communication requires tools for ready access to raw data, and more efficient methods to
communicate the rapidly accumulating sets of scientific information. We propose to leverage
virtual reality (VR) and verbal communication within the VR environment, to streamline each of
these stages of scientific work, by capitalizing on the more natural abilities for stereoscopic
vision and hearing to process scenes and language. Based upon the tool base and direct
volume rendering of large files that we have established in our VR software, called syGlass, we
will first integrate VR into the microscope controls for tuning the microscope and then efficiently
inspecting images in 3D as they are acquired (Aim 1). Next, we will introduce novel domain
adaptation techniques in the ML field to scale up 3D image quantification capabilities for current
acquisition sizes, by coupling them with user-optimized experiences that do not require ML
expertise, and yet provide automated and accurate results (Aim 2). Finally, we will provide tools
to efficiently generate narrated scientific presentations in VR for use in the lab setting, as
manuscript publications, and for production of educational materials (Aim 3). In each of these
activities, we will introduce paradigm shifts in the management of experiments, analysis of the
resulting data, and publication of manuscripts and materials to other scientists and the general
public.

## Key facts

- **NIH application ID:** 10143312
- **Project number:** 5R44MH125238-02
- **Recipient organization:** ISTOVISR
- **Principal Investigator:** Gianfranco Doretto
- **Activity code:** R44 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2021
- **Award amount:** $499,966
- **Award type:** 5
- **Project period:** 2020-05-01 → 2023-01-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10143312

## Citation

> US National Institutes of Health, RePORTER application 10143312, Streamlining Volumetric Imaging, Analysis and Publication Using Immersive Virtual Reality (5R44MH125238-02). Retrieved via AI Analytics 2026-05-24 from https://api.ai-analytics.org/grant/nih/10143312. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*