Project Summary Physicians in the cardiac intensive care unit (CCU) make decisions in an increasingly data- and knowledge- rich world, yet often they get little help. Currently, each physician makes decisions based on his or her mental model of the patient’s physiology, together with mental predictions of the patient’s response to intervention. This approach can lead to a range of behaviors that compromise patient outcomes, including oversimplification of the physiology, errors due to cognitive overload, and physician to physician variability in decision making. A computational tool equipped with quantitative knowledge of physiology, the ability to systematically evaluate all the data, and informed by a database of past action-outcome events could aid the physician with valuable suggestions for action. We propose to train an algorithm to make decisions about dosing vasoactive medications and initiating mechanical support in patients with cardiogenic shock due to decompensated heart failure. This focused set of decisions entails calculations about the physiology that are normally performed in a physician’s head. We frame the decision problem as optimizing cardiovascular function to preserve oxygen delivery, and we apply tools from optimal control. Rather than hand-design a CCU controller we will use reinforcement learning (RL) techniques to “fit” one. The field of RL has experienced explosive growth over the past few years, with notable advances in strategic decision problems and robotics. A key challenge in the clinical environment is that the exploration phase of learning (“trial and error”) would be unethical in real patients. A second challenge is that the availability of patient data, while growing, is likely to be a bottleneck. We will leverage state-of-the-art model-based RL to train an algorithm using a combination of simulation and off-policy learning from historical data. We will use a model of cardiovascular physiology that underlies cardiac simulators in use today for the training of cardiologists. Historical patient data will come from the Massachusetts General Hospital Clinical Data Animation Center which has recorded real-time telemetry waveform data in addition to standard electronic medical record data from all CCU patients spanning several years. This is one of the largest and most complete datasets of its kind. The complexity of managing cardiogenic shock will continue to escalate as tools become more sophisticated and patients live longer, with more extensive comorbidities. Advanced decision support tools could help tame this complexity, improving the quality of care as well as democratizing it.