# Artificial Intelligence Applied to Video and Speech for Objectively Evaluating Social Interaction and Depression in Mild Cognitive Impairment > **NIH NIH R21** · EMORY UNIVERSITY · 2023 · $430,375 ## Abstract Project Summary/Abstract The decreases in social engagement and the feeling of depression and loneliness are reported to be highly correlated with the progression of mild cognitive impairment (MCI). However, previous studies primarily relied on retrospective survey analysis, which lacks continuously quantifying such behavioral biomarkers in the real world. Recent advances in artificial intelligence (AI) made it capable of assessing human physical and mental activities in the real world. Yet, those techniques are mainly evaluated on normal populations, whose behaviors are distinctively different from MCI individuals. This proposal aims to validate AI technologies applied to video and speech for objectively evaluating social interaction and mental health in a MCI population. Successful completion of this proposal will provide an important step toward our long-term goal, which is a large-scale longitudinal study for continuous, objective quantification of MCI progression in the real world. Our general hypothesis is that the subtle yet important changes in social engagement and mental health during real-world interactions of MCI individuals can be continuously captured and quantified for cognitive impairment. For our study, MCI patients will be recruited with balanced race and ethnic backgrounds from the urban Atlanta, GA, area. In Aim 1, social engagement analysis will be conducted at Cognitive Empowerment Program (CEP) at Emory Goizueta Alzheimer’s Disease Research Center. The CEP space is installed with an edge computing-based, privacy- preserving, low-cost patient monitoring system having multi-modal sensors including cameras. At CEP, the recruited MCI patients will be participating in physical and cognitive training programs provided by therapeutic service professionals. MCI patients' social engagement features detected from the multi-camera network system will be used to predict the Perceived Social Support Scale (PSS) and Montreal Cognitive Assessment (MoCA) scores of corresponding patients. In Aims 2 and 3, depression and loneliness analysis is conducted with video recordings of the Cognitive Assessment Interview (CAI) from the recruited MCI patients. For gold standard depression and loneliness scales, MCI participants will provide Geriatric Depression (GDS) and UCLA Loneliness scales, respectively. From the video recordings, the facial (Aim 2) and speech (Aim 3) behavior of MCI patients will be analyzed for the prediction of GDS, UCLA Loneliness, and MoCA scales. The validation of techniques in this proposal is expected to have a significant impact on quantifying MCI progressions as these techniques can be readily extended to quantifying other MCI-related behaviors in the real world, such as wandering or apathy. Also, this proposal’s findings in social and mental health features can drive novel hypothesis generation to power clinical trials for developing novel treatments for cognitive decline. This research aligns with the NIA’s mission to ... ## Key facts - **NIH application ID:** 10810965 - **Project number:** 1R21DC021029-01A1 - **Recipient organization:** EMORY UNIVERSITY - **Principal Investigator:** Gari David Clifford - **Activity code:** R21 (R01, R21, SBIR, etc.) - **Funding institute:** NIH - **Fiscal year:** 2023 - **Award amount:** $430,375 - **Award type:** 1 - **Project period:** 2023-09-25 → 2026-08-31 ## Primary source NIH RePORTER: https://reporter.nih.gov/project-details/10810965 ## Citation > US National Institutes of Health, RePORTER application 10810965, Artificial Intelligence Applied to Video and Speech for Objectively Evaluating Social Interaction and Depression in Mild Cognitive Impairment (1R21DC021029-01A1). Retrieved via AI Analytics 2026-05-26 from https://api.ai-analytics.org/grant/nih/10810965. Licensed CC0. --- *[NIH grants dataset](/datasets/nih-grants) · CC0 1.0*