This course is roughly aimed at early AI PhD or master's students, but it is designed to be accessible to anyone willing to put in the work.
You do not need prior reinforcement learning or language modeling background to start. A motivated learner who studies hard — leaning on today's top LLMs as a tutor to unpack unfamiliar math, code, and jargon — can follow the entire course. I encourage you to go down rabbit holes, skip or reorder videos, and chase what excites you.
If you prefer the traditional coursework path, the usual background is the basics of language modeling / NLP plus basic machine learning (e.g. an intro to AI course and an intro to ML course).
Lecture 0: The ML Foundations of LLM Post-Training
A refresher on the ML prerequisites of post-training — language modeling, KL, cross-entropy, & other math — to acclimate to the series
Conversation 2: From academic research to frontier practice (w/ Scott Geng)
We discuss what it takes to land a well-grounded academic result into a near-frontier model. Scott and I worked together on DPO for Olmo 3, and how building models is much more than the idea, it's making it fit in a more complex puzzle.
Outside material that I've personally used for going deeper on reinforcement learning and language models. The books in particular are wonderful complements.
If you found this useful for your research, please cite it!
For the web and arXiv version:
@misc{lambert2025reinforcementlearninghumanfeedback,
title = {Reinforcement Learning from Human Feedback},
author = {Nathan Lambert},
year = {2025},
eprint = {2504.12501},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2504.12501}
}
For the Manning edition:
@book{lambert2026reinforcement,
author = {Nathan Lambert},
title = {Reinforcement Learning from Human Feedback: Alignment and post-training of {LLMs}},
year = {2026},
publisher = {Manning Publications},
isbn = {9781633434301},
url = {https://www.manning.com/books/reinforcement-learning-from-human-feedback}
}