Summary Artificial intelligence and machine learning (ML) models are becoming increasingly popular in clinical applications. If we allow these “autonomous” ML models to make recommendations for clinical decisions, it is important to ensure that they do introduce algorithmic unfairness (e.g., differences in the burden of disease or opportunities of treatment for different populations). We propose novel technological solutions to mitigate algorithmic unfairness. We will address two major types of data biases (subgroup and representation) to reduce their negative impact on ML models. Based on contextual information and novel causal inference techniques, we will identify potential outliers and task-irrelevant confounders and address them with customized mitigation strategies (e.g., down-sampling and factor reduction) to avoid learning erroneous information that might lead to health disparities. In addition, we will propose FairAUC (a new optimization mechanism) to maximize prediction accuracy while considering fairness by design. As opposed to post-hoc fairness rectification approaches, our method will automatically consider both objectives in the training phase to strike the optimal balance between accuracy and fairness.