Course outline

Course outline#

STAT XXXX Generative AI

This sequence assumes a strong statistics background and begins at an introductory level in deep learning and GenAI. Modules are learning units, not calendar weeks. Work through them at a comfortable pace, returning to the mathematics when it clarifies an experiment. All proposed implementations use Python, progressing from NumPy to PyTorch and then selected pretrained-model tools.

Road map#

Module	Topic	Statistical connection	Evidence of learning
1	Generative AI as probabilistic modeling	Density estimation, likelihood, risk	Derive and sample a tiny language model
2	Python and neural network foundations	GLMs, optimization, regularization	Fit a small neural predictor
3	Tokens and embeddings	Feature maps, dimension reduction	Compare two text representations
4	Attention from first principles	Adaptive weighted averages	Implement and inspect attention
5	The transformer	Compositional representations	Trace a decoder block
6	Training and using language models	Conditional likelihood, sampling	Compare decoding rules
7	Latent-variable generative models	Variational inference, change of variables	Derive an ELBO and simulate
8	Diffusion and score-based modeling	Denoising, stochastic processes	Run a toy noise/recovery experiment
9	Adaptation and alignment	Low-rank constraints, paired comparisons	Compare adaptation strategies
10	Retrieval-augmented generation	Conditional prediction, selection	Build and inspect a small RAG pipeline
11	Evaluation and uncertainty	Experimental design, calibration	Produce an evaluation report
12	Agents and tool use	Sequential decisions, stopping	Build a bounded single agent
13	Reliable agent experiments	Dependence, ablations, reliability	Compare agent and workflow baselines
14	A statistician’s GenAI project	Reproducible empirical research	Deliver a small teaching or research system

Part I · From statistical learning to attention#

1 · Generative AI as probabilistic modeling#

Question. What does a generative model learn, and what does it mean to generate a new observation?

Learn: distinguish joint, conditional, and discriminative models; use the chain rule to factor a text distribution; connect cross-entropy to negative log likelihood.
Statistical lens: density estimation, empirical risk, smoothing, and held-out log loss.
Python practice: estimate a tiny categorical or bigram model, sample sequences, and compare training and held-out loss. Chapter 1 develops the full example, including a bigram language model.
Self-check: explain why fluent sampled text need not be factually correct, and why zero-probability events cause problems for log loss.

2 · Python and neural network foundations#

Question. What changes when a regression model learns its own nonlinear features?

Learn: arrays, shapes, indexing, broadcasting, functions, and reproducible random generators; then tensors, automatic differentiation, minibatches, and a PyTorch training loop.
Statistical lens: linear/logistic regression, basis expansions, regularization, optimization, overfitting, and validation.
Python practice: fit logistic regression and a small multilayer perceptron to the same simulated problem; record train/validation curves and compare against a simple baseline.
Self-check: distinguish the model, loss, optimizer, and data split; check a derivative numerically before trusting autodiff.

3 · Tokens, embeddings, and representations#

Question. How does text become something a numerical model can use?

Learn: tokenization, vocabulary, one-hot representations, embedding lookup, positional information, cosine similarity, and contextual representations.
Statistical lens: design matrices, learned feature maps, matrix factorization, and identifiability.
Python practice: tokenize a small public text collection and compare bag-of-words with embedding-based similarity. Inspect examples where similarity is misleading.
Self-check: explain why a token is not necessarily a word and why a two-dimensional embedding plot is not evidence of causal structure.

4 · Attention from first principles#

Question. How can a representation select and combine information relevant to a query?

Learn: queries, keys, values, scaled dot products, row-wise softmax, self-attention, cross-attention, causal masks, and multiple heads.
Statistical lens: a query-dependent normalized weighted average; compare with kernel smoothing without claiming exact equivalence.
Python practice: calculate a three-token attention example by hand, implement it in NumPy, verify dimensions and row sums, and plot how weights change when keys change.
Self-check: explain the role of the mask and the \(\sqrt{d_k}\) scaling; describe why attention weights alone are not a faithful explanation of a prediction.
Reading: Attention Is All You Need, especially the architecture and attention equations.

Part II · Transformers and generative models#

5 · Assembling the transformer#

Question. How does one attention calculation become a sequence model?

Learn: residual connections, normalization, feed-forward layers, positional encodings, encoder/decoder distinctions, and a decoder-only forward pass.
Statistical lens: a composition of fitted feature transformations with shared parameters and a conditional prediction head.
Python practice: implement a tiny decoder block in PyTorch; track every tensor’s dimensions and use a causal-mask check to show future tokens cannot influence earlier outputs.
Self-check: draw the route from token IDs to next-token probabilities, and distinguish the original encoder-decoder transformer from a decoder-only language model.

6 · Language model training, prompting, and decoding#

Question. What is learned in the weights, and what changes when we provide a prompt?

Learn: next-token prediction, teacher forcing, context windows, pretraining versus inference, in-context examples, greedy decoding, temperature, top-k, and top-p.
Statistical lens: conditional likelihood, distribution shift, and transformations of a sampling distribution. In-context behavior can be studied statistically without assuming the model performs exact Bayesian inference.
Python practice: train a very small character model or use a small pretrained model; vary one decoding choice at a time on fixed prompts and repeat runs.
Self-check: explain why perplexity is not a universal measure of usefulness and why prompting is not parameter estimation by gradient descent at inference time.

7 · Latent variables, VAEs, GANs, and flows#

Question. What alternatives to sequential factorization can represent a complex distribution?

Learn: the encoder/decoder view of a VAE, the evidence lower bound (ELBO), reparameterization, and a conceptual comparison with adversarial learning and invertible flows.
Statistical lens: latent-variable models, variational inference, density-ratio ideas, and change of variables. Compare objectives, sampling, and access to likelihoods.
Python practice: derive the ELBO and implement a small VAE on synthetic two-dimensional data. Treat GANs and flows as comparison topics on the first pass.
Self-check: separate the latent prior, approximate posterior, and observation model; explain why matching a few summary statistics cannot establish distributional equivalence.
Reading: Auto-Encoding Variational Bayes.

8 · Diffusion and score-based generation#

Question. How can learning to remove noise support generation from noise?

Learn: a forward noising process, learned reverse transitions, denoising objectives, score functions, and conditional generation; preview image and multimodal applications.
Statistical lens: stochastic processes, denoising estimation, and the score \(\nabla_x\log p(x)\), which differs from a likelihood score with respect to model parameters.
Python practice: simulate progressive Gaussian noising on a mixture distribution, then study a small learned denoiser and the effect of the noise schedule.
Self-check: distinguish the known forward process from the fitted reverse process and separate training from sampling.
Reading: Denoising Diffusion Probabilistic Models. Continuous-time formulations are an optional extension.

9 · Fine-tuning, LoRA, and preference learning#

Question. How do we adapt an existing model, and whose preferences are represented?

Learn: supervised fine-tuning, parameter-efficient updates, LoRA, instruction data, preference data, the RLHF pipeline, and the basic DPO idea.
Statistical lens: transfer learning, regularization, low-rank parameter updates, and Bradley–Terry-style paired comparisons. Keep model quality separate from preference-label quality.
Python practice: fit a toy paired-comparison model and compare a full matrix update with a low-rank update. Small-model fine-tuning is optional after the conceptual exercise.
Self-check: explain what changes under prompting, RAG, and fine-tuning; identify label noise and selection bias in preference data.
Readings: LoRA and DPO.

Part III · Reliable systems and agents#

10 · Retrieval-augmented generation (RAG)#

Question. How can a model answer using a specified collection of documents?

Learn: document chunking, indexing, retrieval, reranking, context assembly, citation, and abstention when evidence is insufficient.
Statistical lens: prediction conditional on selected evidence, retrieval error, measurement error, and the interaction between pipeline components.
Python practice: build a small pipeline over public statistical notes. Inspect retrieved passages, compare with a keyword baseline, and test answerable and unanswerable questions separately.
Self-check: identify whether an error arose in retrieval, evidence selection, or generation. A citation does not by itself show that its source supports the claim.
Reading: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.

11 · Evaluation, calibration, and uncertainty#

Question. What evidence would justify saying one system is better than another?

Learn: task-specific metrics, held-out evaluation, data contamination, human annotation, LLM-as-a-judge limitations, calibration, abstention, and an introduction to conformal methods.
Statistical lens: estimands, sampling units, paired comparisons, bootstrap uncertainty, clustered dependence, and distribution shift. Conformal coverage needs an explicit target and assumptions such as exchangeability; it is not an automatic guarantee of free-text truthfulness.
Python practice: compare two fixed systems on the same held-out questions; bootstrap at the question level, inspect judge disagreement, and draw a risk-versus-coverage curve for abstention.
Self-check: distinguish token probabilities, answer confidence, and factual correctness; state the population to which a reported interval applies.
Reading: A Gentle Introduction to Conformal Prediction.

12 · Agents: tools, state, and decisions#

Question. What makes a system an agent, beyond producing a text response?

Learn: a model–action–observation loop, tool schemas, state, memory, structured outputs, retries, and stopping rules; distinguish a fixed workflow from adaptive action selection.
Statistical lens: sequential decision making, partial observation, expected utility, and the value of information. Learn the basic state/action/reward vocabulary without requiring a reinforcement learning course first.
Python practice: build a single agent with two narrow tools, such as a calculator and document lookup. Validate arguments, log actions and observations, and enforce a small step budget.
Self-check: explain when a fixed Python function or workflow is sufficient and why tool access does not imply reliable tool use.
Reading: ReAct; study observable actions and results without treating generated reasoning text as a faithful record of internal computation.

13 · Agent reliability and experimental design#

Question. Does an agent improve outcomes enough to justify its additional complexity?

Learn: trajectory-level evaluation, failure taxonomies, repeated trials, cost/latency tradeoffs, memory ablations, prompt injection through retrieved text, and human approval for consequential actions.
Statistical lens: dependence within a task trajectory, factorial comparisons, uncertainty in success rates, and adaptive selection bias.
Python practice: compare a direct model call, a fixed workflow, and the agent on the same task set and budget. Inject harmless tool failures; log recovery, loop counts, and stopping behavior. Treat external documents as data rather than tool instructions.
Self-check: separate task failure, tool failure, unsupported answers, and budget exhaustion. Add multiple agents only as an optional ablation after establishing a single-agent baseline.

14 · Capstone: a GenAI system for a statistician#

Choose one modest project: a citation-grounded assistant for public statistical notes, a teaching assistant that explains simulation results, or a tool-using simulation planner with a constrained action space.

Produce a reproducible Python notebook and a short MyST chapter containing:

A target user, task population, loss, and explicit scope.
A simple baseline and one justified extension, such as RAG or tool use.
Separate development and held-out evaluation sets, with provenance.
Repeated-run results, task-level uncertainty, cost, and failure examples.
An ablation that tests what the added component contributes.
A plain-language explanation of what the evidence supports and where the system fails.

Success means being able to explain, implement, evaluate, and teach the system. A small, well-understood experiment is a complete capstone.

Optional directions after the core#

Multimodal models, advanced diffusion/SDE theory, scaling laws, interpretability, reasoning-model training, causal questions about evaluation, and multi-agent systems can each become a later topic. Select them to answer a concrete research question rather than trying to learn every framework at once.