# Identifying mechanisms of multisensory memory using virtual reality and fMRI

> **NIH NIH F31** · UNIVERSITY OF CALIFORNIA AT DAVIS · 2023 · $40,288

## Abstract

Project Summary
Forming and retrieving memories for objects and events is fundamental to human experience. Complementary
mechanisms within the medial temporal lobe (MTL) support the formation and retrieval of memories for objects,
such that the perirhinal cortex supports memory for features of individual items, and the hippocampus supports
recollection of the object within the context it was originally encountered. Although most research on object
memory has been conducted using visual stimuli alone, a small body of research has shown that visual objects
encoded along with their characteristic sound (e.g., a dog and a bark) are better remembered later on than
objects encoded only visually, or with a meaningless or non-characteristic sound. Despite this evidence and
the ubiquity of multimodal stimuli within natural environments, our knowledge of the extent of multisensory
influences on memory and its underlying mechanisms is very limited. My recent behavioral work showed that
the benefits of multisensory encoding are based on improved recollection of the context in which those objects
were encoded, suggesting that multisensory processing uniquely engages mechanisms that bind these objects
to surrounding information at encoding. However, it remains unclear what features of the encoding context are
better retrieved, and what neural mechanisms are modulated by multisensory processing to allow for this
benefit. The proposed research addresses this gap in knowledge by collecting the critical evidence to
determine the facets of memory for items and surrounding events that are influenced by multisensory
processing, and the brain activity patterns and regions that are involved in retrieving such information. We
hypothesize that multisensory processing at encoding improves object memory by increasing the likelihood
that the object and context will be bound into an episodic memory to support later recognition of that object,
rather than by enhancing memory for the individual object itself. Further, we predict that patterns of activation
within the hippocampus during retrieval of multimodal objects will also carry information about the environment
in which these items were encoded. To rigorously test the type of memory that multisensory processing
impacts, there is a need to assess memory for unimodal and multimodal objects that are embedded within rich,
naturalistic spatiotemporal context. To this end, we have developed an immersive, naturalistic encoding task
within virtual reality (VR) environments, which contain controlled but animated visual and audiovisual objects.
This novel approach will allow us to assess whether multisensory processing influences memory for individual
objects alone or if these objects are more readily bound to their surroundings to support memory for events
within context (Aim 1). Further, analyses of fMRI data will be used to determine which regions of the MTL and
cortex are specifically involved in memories of multisensory object...

## Key facts

- **NIH application ID:** 10676058
- **Project number:** 1F31MH133270-01
- **Recipient organization:** UNIVERSITY OF CALIFORNIA AT DAVIS
- **Principal Investigator:** Shea Duarte
- **Activity code:** F31 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2023
- **Award amount:** $40,288
- **Award type:** 1
- **Project period:** 2023-09-10 → 2026-12-11

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10676058

## Citation

> US National Institutes of Health, RePORTER application 10676058, Identifying mechanisms of multisensory memory using virtual reality and fMRI (1F31MH133270-01). Retrieved via AI Analytics 2026-05-25 from https://api.ai-analytics.org/grant/nih/10676058. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*