Pretrained generative language models (LMs) have become popular labor-saving tools in domains like social media, education, business, and medicine. However, while these models have proven useful for generative tasks like programming and writing, it has been difficult to effectively use them as decision-making assistants for tasks like application review or grading. The key problem is verification: these models are opaque and prone to "hallucination" (i.e., making up content), logical mistakes, and other errors, making it hard to verify their advice. Existing research in the field of explainable artificial intelligence (XAI) has sought to solve the problem by exposing the underlying model logic for human scrutiny but has struggled to demonstrate improvements in decision-making performance. This project will develop a new approach to the verification problem in LM-supported decision-making by focusing on settings where existing guidelines (such as grading rubrics, job listings, and medical guidelines) describe how decisions should be made. The team will develop methods for using LMs to apply existing written guidelines to decision-making tasks, and present explanations of recommended decisions that can be verified in terms of those guidelines. The idea is that this approach will lead to more useful explanations and better decisions than existing XAI approaches in practical domains, advancing the science of explainable AI while having beneficial impacts on many societal problems