Interpersonal behavioral synchrony in virtual and in-person dyadic conversation

NIH RePORTER · NIH · R34 · $364,647 · view on reporter.nih.gov ↗

Abstract

PROJECT SUMMARY/ABSTRACT Human dyadic social communication entails a rich repertoire of expression, including not only face expression (and gaze), but also acoustics (prosody and pauses) turn-taking, gestures and language. Communication has evolved in humans within a social context, beginning with the parent-infant dyad, with mirroring of facial expressions and sounds. Its natural ecology is face-to-face dyadic interactions, both in-person and increasingly via remote platforms for teleconferencing and telehealth. Social communication is a “complex orchestration” in real time: its signals are multiple and temporally offset. It is a continuous exchange that is highly coordinated between speakers, with norms for turn-taking and alignment of face expression, gesture, semantic content and speech rates. As yet, a critical gap exists in that we lack the tools to quantify and analyze temporal patterns of multimodal communication behavior between two individuals in face-to-face communication, in an ecologically valid setting, that have the same rigor and reproducibility as do hyperscanning approaches to record brain activity during dyadic conversation. This tool must be developed to realize the true potential of second-person neuroscience. This planning proposal for tool development entails several key activities, beginning with the convening of a diverse multidisciplinary team of experts from various fields, including ethics/regulatory, anthropology, cognitive neuroscience, computer science, engineering, physics, mathematics, psychiatry and neurology. This team will discuss ethics, diversity, paradigm development, and computational frameworks, and providing iterative feedback and convening also with advocacy groups. Also, we will build two testing rooms for multimodal recording of dyadic communication, to demonstrate feasibility of acquiring and synching high temporal resolution data. Pilot EEG hyperscanning will be done concurrently in a subcohort. Further, given increased use of teleconferencing, dyadic communication data will be collected via remote platform and compared with in-person data, to determine how information may be degraded by differences in resolution and streaming delays. We will also develop computational frameworks for analyses of multimodal data.

Key facts

NIH application ID
10797870
Project number
1R34DA059716-01
Recipient
ICAHN SCHOOL OF MEDICINE AT MOUNT SINAI
Principal Investigator
CHERYL MARY CORCORAN
Activity code
R34
Funding institute
NIH
Fiscal year
2024
Award amount
$364,647
Award type
1
Project period
2024-08-15 → 2026-07-31