Reading guide#
Start with the introductory resources. Read foundational papers after working through the corresponding small example. This is a selected learning path, not an exhaustive survey or a claim about the newest model.
First-pass resources#
Resource |
When to use it |
Reading goal |
|---|---|---|
Modules 2–5 |
Work through arrays, autodiff, multilayer perceptrons, attention, and transformers; skip familiar mathematical preliminaries as needed. |
|
Module 2 onward |
Learn tensors, datasets, model definitions, gradients, and the training loop. |
|
Modules 3, 5, 6, and 9 |
Connect tokenization and transformer concepts to practical model tooling. |
|
Throughout |
Write and build executable teaching notes with Python. |
Foundational papers by module#
Attention and transformers — Module 4–5: Vaswani et al. (2017), Attention Is All You Need. First identify the inputs, outputs, attention equation, and block diagram; return to experiments later.
Latent-variable generation — Module 7: Kingma and Welling (2013 preprint), Auto-Encoding Variational Bayes. Connect the ELBO to variational inference and study the reparameterization step.
Diffusion — Module 8: Ho, Jain, and Abbeel (2020), Denoising Diffusion Probabilistic Models. Track the forward and reverse processes and the denoising objective.
Efficient adaptation — Module 9: Hu et al. (2021 preprint), LoRA: Low-Rank Adaptation of Large Language Models. Identify which parameters are frozen and which low-rank matrices are learned.
Preference learning — Module 9: Rafailov et al. (2023), Direct Preference Optimization: Your Language Model is Secretly a Reward Model. Study the preference likelihood and the role of the reference policy.
Retrieval — Module 10: Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Identify the retriever, generator, and conditioning information; distinguish the original model from a simple modern retrieval pipeline.
Uncertainty — Module 11: Angelopoulos and Bates (2021 preprint), A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification. Establish the prediction target, calibration set, and coverage assumptions before applying the ideas to generated outputs.
Agents — Module 12: Yao et al. (2022 preprint; ICLR 2023), ReAct: Synergizing Reasoning and Acting in Language Models. Follow the interaction with tools and observations, and inspect how success is measured.
A statistician’s reading note#
For each paper, write a short note answering: What is the statistical object? What is optimized? What assumptions are made? What is the comparison baseline? What are the experimental units? Which claim would I want to reproduce?
Links and source identities were checked when this outline was created on October 20, 2026. Software documentation evolves; record the versions used for each future lab.