Prospective Students

Course Staff

Instructor (Guest) Co-instructor (Guest) Co-instructor
Dawn Song Xinyun Chen Kaiyu Yang
Professor, UC Berkeley Research Scientist,
Google DeepMind
Research Scientist,
Meta FAIR

Course Description

Large language model (LLM) agents have been an important frontier in AI, however, they still fall short critical skills, such as complex reasoning and planning, for solving hard problems and enabling end-to-end applications in real-world scenarios. Building on our previous course, this course dives deeper into advanced topics in LLM agents, focusing on reasoning, AI for mathematics, code generation, and program verification. We begin by introducing advanced inference and post-training techniques for building LLM agents that can search and plan. Then, we focus on two application domains: mathematics and programming. We study how LLMs can be used to prove mathematical theorems, as well as generate and reason about computer programs. Specifically, we will cover the following topics:

Syllabus

Date Guest Lecture
(4:00PM-6:00PM PST)
Supplemental Readings
Jan 27th Inference-Time Techniques for LLM Reasoning
Xinyun Chen, Google DeepMind
Livestream Intro Slides Quiz 1
- Large Language Models as Optimizers
- Large Language Models Cannot Self-Correct Reasoning Yet
- Teaching Large Language Models to Self-Debug
Feb 3rd Learning to reason with LLMs
Jason Weston, Meta
Livestream Slides Quiz 2
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- Iterative Reasoning Preference Optimization
- Chain-of-Verification Reduces Hallucination in Large Language Models
Feb 10th On Reasoning, Memory, and Planning of Language Agents
Yu Su, Ohio State University
- Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
- HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
- Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents
Feb 17th No Class - Presidents’ Day  
Feb 24th Reasoning and Planning in Large Language Models
Hanna Hajishirzi, University of Washington
 
Mar 3rd Coding Agents and AI for Vulnerability Detection
Charles Sutton, Google DeepMind
 
Mar 10th Coding agents/web agents
Ruslan Salakhutdinov, CMU/Meta
 
Mar 17th Multimodal Agents
Caiming Xiong, Salesforce AI Research
 
Mar 24th No Class - Spring Recess  
Mar 31st AlphaProof
Thomas Hubert, Google DeepMind
 
Apr 7th Language models for autoformalization and theorem proving
Kaiyu Yang, Meta FAIR
 
Apr 14th Advanced Topics in Neural Theorem Proving
Sean Welleck, CMU
 
Apr 21st Program verification & generating verified code
Swarat Chaudhuri, UT Austin
 
Apr 28th Agent safety & security
Dawn Song, UC Berkeley
 

Completion Certificate

Coming Soon!