ABSTRACT Modern medical imaging systems comprise complicated hardware and sophisticated computational methods. Given the sheer number of system parameters that impact image quality, the large variety in objects to be imaged, and ethical concerns, the assessment and refinement of emerging imaging technologies via clinical trials often is impossible. For these reasons, there is great interest in virtual imaging trials (VITs) that permit the automated simulation and analysis of clinically relevant imaging experiments. During the development and refinement of new imaging technologies via VITs, there is an important need for assessing objective image quality measures (OIQMs) that quantify the best possible utility of the resulting images for different diagnostic tasks—independent of the ability of the observer (human or algorithm) who interprets the images. In effect, such OIQMs can reveal the extent to which task-related information is present in imaging data and thus can be potentially extracted by a human observer or other numerical algorithm that is sub-optimal; this can permit the identification of opportunities for improved image processing or other technology changes that lead to improved performance on diagnostic tasks. The broad objective of the proposed research is to address this challenge by developing the next generation of open source and modality-agnostic computational methods for computing OIQMs that quantify the best possible performance of an imaging system—the so-called ideal observer performance—for clinically relevant tasks. Estimation of the best achievable performance of medical imaging technologies using realistic stochastic digital object phantoms and clinically relevant diagnostic tasks has been a holy grail for the medical image-quality assessment field, and the lack of success to date has limited the field to unrealistic object models and tasks for decades. When employed in VITs, our new methods will permit assessment of the amount of task-relevant information in image data and will accelerate the refinement and translation of promising new imaging technologies to the clinic. The Specific Aims of the project are: Aim 1: To develop and validate ambient generative adversarial networks (AmGANs) for creating ensembles of clinically relevant digital phantoms; Aim 2: To develop methods for estimating the optimal task performance of an imaging technology; Aim 3: To use the developed tools for assessing deep learning-based image restoration. The developed computational tools for computing OIQMs will be made open source. This will open entirely new avenues for assessing and refining emerging medical imaging technologies with a level of rigor and clinical relevance previously not possible.