# SCH: An Augmented Reality Neurorehabilitation System for Monitoring and Management of Motor Symptoms of Parkinson's Disease

> **NIH NIH R01** · UNIV OF NORTH CAROLINA CHAPEL HILL · 2024 · $291,816

## Abstract

PROJECT SUMMARY (See instructions): 
This project proposes augmented reality eyeglasses that can be worn throughout the day by a person with 
Parkinson's disease (PD) undergoing neurorehabilitation. The system enables (1) monitoring of and 
feedback to the patient during clinic visits, with provider-supervised physical therapy (PT) exercises, (2) 
display of immersive guidance and feedback to the patient during PT exercises at home that reference 
exercises supervised in the clinic, and (3) monitoring, analysis, and summarization of the patient’s activities 
of daily life (ADLs) between clinic visits, through cameras integrated into the eyeglasses and inertial 
sensors in shoes and wristbands. To develop such a system, we will advance the state-of-the-art in four 
research thrusts - egocentric tracking and reconstruction, egocentric visual perception, personalized 
reasoning models, and immersive analytics. In the first thrust, we will investigate the use of deep neural 
network architectures for highly accurate, clinically useful egocentric tracking and 3D reconstruction, 
enabling us to meaningfully record, playback, and evaluate rehabilitation activities. In the second thrust, we 
will develop a rich semantic understanding of contextualized activities using multimodal deep learning 
models that take into account short-term and long-term data from wearable inertial sensors and multiple 
egocentric cameras. In the third thrust, we will develop novel reasoning systems based on graphical 
models that leverage techniques such as partially observable Markov decision processes (POMDPs) to 
track and predict aspects of user performance during rehabilitation activities. In the fourth thrust, we will 
design an immersive analytics front-end interface that will integrate with our reasoning systems to provide 
customized user feedback and clinician summaries, enabling improved patient care. Developments across 
each of these thrusts will be integrated into an eyeglasses-based augmented reality system that will be 
iteratively evaluated and refined, first with able-bodied participants in a controlled lab setting, next with 
Parkinson’s patients in a clinical setting, then with able-bodied participants in home settings, and finally with 
Parkinson’s patients in their home environment.

## Key facts

- **NIH application ID:** 10894930
- **Project number:** 5R01HD111074-03
- **Recipient organization:** UNIV OF NORTH CAROLINA CHAPEL HILL
- **Principal Investigator:** GEDIMINAS Bertasius
- **Activity code:** R01 (R01, R21, SBIR, etc.)
- **Funding institute:** NIH
- **Fiscal year:** 2024
- **Award amount:** $291,816
- **Award type:** 5
- **Project period:** 2022-08-01 → 2026-07-31

## Primary source

NIH RePORTER: https://reporter.nih.gov/project-details/10894930

## Citation

> US National Institutes of Health, RePORTER application 10894930, SCH: An Augmented Reality Neurorehabilitation System for Monitoring and Management of Motor Symptoms of Parkinson's Disease (5R01HD111074-03). Retrieved via AI Analytics 2026-06-11 from https://api.ai-analytics.org/grant/nih/10894930. Licensed CC0.

---

*[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*