Johnson, Kevin DP1 Details Project Name: Helping Doctors Doctor: Using AI to Automate Documentation and “De-Autonomate” Health Care Project Summary Clinical encounter documentation is one of the most time-consuming tasks of the ambulatory encounter, taking approximately two hours for every hour spent with patients. Clinical note lengths in the US are longer than those in other countries due to requirements to justify billing and complete quality care metrics. Not surprisingly, clinical encounter documentation has become a major source of clinician burnout over the past two decades. Although companies are selling technologies to transcribe what was discussed during the encounter, these costly solutions reproduce what is already suboptimal about how we create and use EHR information. There is a desperate need to reimagine the process of documenting as well as the content of documenting a clinical encounter. In this application I propose to develop a new generation of automated documentation algorithms—algorithms that can listen to the dialog between a patient and clinician, collect quantitative data about these observations, combine those with existing electronic health record (EHR) data and create relevant encounter summary information. These documentation algorithms will leverage the remarkable progress we have made in computer vision, natural language processing, machine learning to support image labeling, and other advances using EHR data. These novel computational approaches have yet to be explored as alternative approaches to summarizing medical data collected in real time. As a pediatrician and biomedical informatician who has acquired considerable expertise in real-world systems design, implementation and medical data analytics, this project leverages many of my skills, though it is a departure from my previous human-computer interface work. Rather, the goal of this project is to remove the burden of documentation from clinicians to the extent possible. To achieve this goal, I will work with a multidisciplinary group of collaborators, including computer scientists, technology engineers, and clinical domain experts. Specifically, I will: (1) collect and analyze exam room video and annotations of the encounters to identify salient characteristics of patients and their interaction with the clinician that led to specific diagnoses; (2) apply natural language processing, deep learning, and computer vision methods to learn and characterize patterns from vast streams of data using supervised and unsupervised learning methods. Combining these techniques to directly impact what is documented and how it is generated is a new area of investigation for me and an approach that promises to support innumerable other projects including identifying implicit bias in clinical encounters, enabling a new class of real-time decision making and improving the usefulness of encounter summaries.