Generative AI through a statistical lens#
Topic Course · Independent study
From probability to attention.
From models to agents.
A personal learning course by Chi-Kuang Yeh, beginning with statistics and statistical machine learning, and learning modern generative AI from the ground up.
Begin with what you know#
You understand probability, inference, linear algebra, optimization, and statistical machine learning. This course uses that background to explain generative AI, attention, transformers, and agents. No prior Python or GenAI knowledge is assumed. We introduce the programming language alongside the mathematics.
This is a self-paced Topic Course for independent study, with material that can later be adapted for teaching. There is no fixed semester or grading. The lessons are in English, using the terminology of the research literature.
Start with the complete first chapter
Chapter 1: From probability models to generative AI develops categorical estimation, log loss, smoothing, sampling, temperature, and a working bigram language model. It includes explained Python code, executed outputs, eight exercises, worked solutions, and a downloadable notebook.
A course in three stages#
Stage |
Chapters |
What you will understand |
|---|---|---|
Model the data |
1–4 |
Probability models, Python, embeddings, and attention as a context-dependent weighted average. |
Understand generation |
5–10 |
Transformers, language models, latent variables, diffusion, adaptation, and retrieval. |
Evaluate the system |
11–14 |
Uncertainty, agents, reliable experiments, and a small teaching or research project. |
Every chapter starts with a concrete question, develops a statistical formulation, and connects it to a small Python experiment. NumPy makes the mathematics visible; later chapters introduce PyTorch for neural networks.
Study and experiment#
Set up Python and JupyterLab. You can also read the rendered lessons without installing anything.
Work through Chapter 1, running cells from top to bottom.
Use the statistical translation guide and selected readings when terminology is unfamiliar.
Follow the 14-chapter course outline as the remaining lessons develop.
Current materials: Chapter 1 is a complete lesson. Chapters 2–14 have individual pages with questions, learning objectives, statistical connections, and planned Python practice; they are explicitly marked as outlines. The short Python warm-up remains available as a supplement.
Black and yellow recall Waterloo; pink celebrates my Faculty of Mathematics background. The palette follows Waterloo's published colour guide. This is a personal course website.