This project will study how artificial intelligence (AI) models can learn new tasks. Today, AI models are often taught to perform a task (e.g., controlling a self-driving car) by showing them examples of what a human would do. This project builds upon an area of research known as reinforcement learning, where AI models learn by trial and error, much like a dog might learn a trick by trying different behaviors and receiving a treat for the correct one. A major challenge in existing algorithms for trial-and-error learning is that complex tasks, such as assembling a house with a robotic arm, may require hundreds of small steps to complete. If the AI model receives feedback (a success or failure signal) only after completing the entire task, then it is difficult to figure out what went wrong in all the small intermediate steps. This project takes three key steps to address this challenge. First, the research will develop new algorithms to discover small, reusable skills. For a robotic arm, instead of teaching it how to build an entire house at once, the AI model might first learn a skill for stacking blocks, then another for sorting blocks, and so on. Importantly, the AI model discovers these skills by exploring and experimenting, without requiring human demonstrations or hand-written code. Once learned, these skills can be rapidly combined and adapted to solve new, more complex tasks. Second, the research will create new simulators and algorithms that leverage GPUs to significan