A · Python and PyTorch, one operation at a time

A · Python and PyTorch, one operation at a time#

For a statistician learning a new programming language. You do not need to relearn regression to learn Python. You do need to recognize which objects hold data, which operations transform them, and which instructions change fitted parameters.

Download the executed notebook · Installation

Read A.1–A.4 alongside Chapter 1 and before Chapter 2. Read A.5–A.7 before the neural-network lab in Chapter 2, A.8 before Chapter 5, and A.9 before Chapters 10, 12, and 14. A.10 is practice, not an additional prerequisite. A.11 explains local imports and the live-notebook workflow; A.12 explains compact syntax used later. Run each cell in order. Predict its output before pressing Shift+Enter. The examples are deliberately smaller than the chapter models; once each operation is familiar, return to the full experiment.

A.1 · Names, values, and containers#

An assignment name = expression evaluates the right side and associates its result with the name. It is not a mathematical equality constraint. Python indexes from zero. Parentheses call a function, brackets select an element, and braces construct a dictionary or set.

Code

n = 4                         # integer: a count
alpha = 0.05                  # float: an approximate real number
label = 'pilot'               # string: text
values = [2.0, 4.0, 6.0, 8.0] # list: an ordered container
request = {'dataset': label, 'n': n}  # dictionary: keys map to values
print(type(n).__name__, type(alpha).__name__, type(label).__name__)
print(values[0], values[-1])   # first and last items
print(values[1:3])             # positions 1 and 2; the stop index is excluded
print(request['dataset'])     # use a dictionary key, not a numeric position

Output

int float str
2.0 8.0
[4.0, 6.0]
pilot

# begins a comment. print() displays a value; it does not save a model. A string such as '4' is text, while 4 is an integer. The Boolean values True and False represent logical conditions. None means no value has been supplied; it is different from zero.

Lists have reference semantics. Copy before an independent modification:

Code

original = [2, 4]
alias = original
independent = original.copy()
alias.append(6)
print('original:', original, 'independent copy:', independent)

Output

original: [2, 4, 6] independent copy: [2, 4]

Both alias and original refer to the same list. This is why a saved model checkpoint needs independent tensors, not just another name for tensors that training will change.

A.2 · Functions, indentation, and small loops#

A function declares its inputs after def. The indented body runs when the function is called. return sends a result back to the caller. A default value is used only when that argument is omitted.

Code

def centered(numbers, center=0.0):
    result = []
    for number in numbers:
        result.append(number - center)
    return result

shifted = centered(values, center=5.0)
print(shifted)

Output

[-3.0, -1.0, 1.0, 3.0]

Read the loop literally: take each number, subtract the center, append the result. The equivalent list comprehension is shorter, but not a different algorithm:

Code

compact = [number - 5.0 for number in values]
assert compact == shifted
for index, number in enumerate(values):
    print(index, number)

Output

2.0
4.0
6.0
8.0

enumerate() supplies both index and value. assert raises an error when its condition is false. Use assertions for diagnostic invariants; use explicit exceptions for public input validation. A notebook kernel retains objects between cells. Restart and run all cells to discover hidden dependence on an earlier experiment.

A.3 · Arrays, shapes, and axes#

A NumPy array stores numerical values with a common data type. Import a library once, and use its short name as a prefix. shape is a tuple describing axis lengths. Rows usually index observations, but that convention is ours: the computer does not know what an observation is.

Code

import numpy as np
X = np.array([[1., 2.], [3., 4.], [5., 6.]])
print('X shape:', X.shape)
print('first observation:', X[0, :])
print('first variable:', X[:, 0])
print('column means:', X.mean(axis=0))
print('row means:', X.mean(axis=1))

Output

X shape: (3, 2)
first observation: [1. 2.]
first variable: [1. 3. 5.]
column means: [3. 4.]
row means: [1.5 3.5 5.5]

An operation along axis=0 reduces the row axis, leaving one answer per column. axis=1 reduces the column axis. A colon selects all entries of that axis. X[:, 0] has shape (3,); X[:, :1] has shape (3, 1). Those shapes can behave differently in arithmetic.

Code

prediction = np.array([[0.2], [0.8]])
target = np.array([0., 1.])
wrong = prediction - target
right = prediction[:, 0] - target
print('unexpected residual matrix:', wrong, 'shape:', wrong.shape)
print('intended residual vector:', right, 'shape:', right.shape)
assert right.shape == target.shape

Output

unexpected residual matrix: [[ 0.2 -0.8]
 [ 0.8 -0.2]] shape: (2, 2)
intended residual vector: [ 0.2 -0.2] shape: (2,)

Broadcasting compares shapes from the right. Dimensions must be equal or one. (2,1) and (2,) therefore produce (2,2): the code runs, but the estimand has changed. Check shapes before averaging a loss. * multiplies elementwise; @ is matrix multiplication.

Code

beta = np.array([0.5, -0.25])
linear_predictor = X @ beta
print(linear_predictor, linear_predictor.shape)
assert np.allclose(linear_predictor, (X * beta).sum(axis=1))

Output

[0.  0.5 1. ] (3,)

allclose() compares floating-point numbers within tolerances. Exact == is useful for integer identities but often inappropriate for numerically equivalent real-valued calculations.

A.4 · Random generators and reproducible simulation#

A generator stores a pseudorandom state. A seed initializes it; each subsequent draw advances it. Recreating a generator inside every replicate with the same seed would repeat the same sample.

Code

rng = np.random.default_rng(2026)
first = rng.normal(size=3)
second = rng.normal(size=3)
restarted = np.random.default_rng(2026).normal(size=3)
print('first:', first.round(3))
print('next:', second.round(3))
assert np.array_equal(first, restarted)
assert not np.array_equal(first, second)

Output

first: [-0.793  0.241 -1.896]
next: [ 1.396  0.638 -0.292]

Reproducibility lets us inspect a computation. It does not establish external validity, independence of test tasks, or robustness across training seeds. .round(3) changes the displayed numerical values here; keep unrounded arrays for fitting and scoring.

A.5 · Tensors and one derivative#

A PyTorch tensor resembles a NumPy array but can record a computation graph. requires_grad=True asks PyTorch to track derivatives with respect to that tensor. For squared-error regression, you already know the derivative; use it to check the software.

Code

import torch
from torch import nn
torch.set_num_threads(1)
torch.manual_seed(2026)
x = torch.tensor([1., 2., 3.])
y = torch.tensor([2., 4., 6.])
beta_t = torch.tensor(0.5, requires_grad=True)
predicted = x * beta_t
loss = ((predicted - y) ** 2).mean()
loss.backward()
analytic = (2 * x * (predicted - y)).mean()
print('loss:', loss.item(), 'autodiff:', beta_t.grad.item())
assert torch.allclose(beta_t.grad, analytic)

Output

loss: 10.5 autodiff: -14.0

backward() propagates derivatives to leaf parameters and stores them in .grad. .item() extracts a Python scalar for reporting. .detach() produces a tensor view without gradient tracking; .detach().cpu().numpy() converts a tensor to a NumPy array for analysis. Do not detach the loss before backward(): that would sever the derivative path.

Gradients accumulate until cleared. That allows deliberate accumulation over batches, but an ordinary training step must clear the preceding step’s gradient.

A.6 · What a model class means#

A class groups state and operations. An instance is one concrete object made from the class. self means this instance. __init__ runs at construction. super().__init__() initializes PyTorch’s parent class, allowing it to register parameters and submodules. forward() declares the prediction calculation.

Code

class OneFeatureRegression(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(1, 1)

    def forward(self, inputs):
        return self.linear(inputs)

model = OneFeatureRegression()
inputs = x.reshape(3, 1)
targets = y.reshape(3, 1)
print('input:', inputs.shape, 'output:', model(inputs).shape)
for name, parameter in model.named_parameters():
    print(name, tuple(parameter.shape))

Output

input: torch.Size([3, 1]) output: torch.Size([3, 1])
linear.weight (1, 1)
linear.bias (1,)

nn.Linear(1,1) stores a slope and intercept. Calling model(inputs) invokes the module machinery and then forward; use this form rather than calling forward directly. The weights are initially random. A model object specifies a family and holds parameter values; creating it does not fit it.

A.7 · Perform one update, then build a loop#

The optimizer receives references to the model parameters. Its learning rate controls the size of an update. It does not choose our statistical loss or data split.

Code

optimizer = torch.optim.SGD(model.parameters(), lr=0.05)
optimizer.zero_grad()                   # 1. erase old gradients
predicted = model(inputs)               # 2. calculate fitted values
loss = ((predicted - targets) ** 2).mean()  # 3. evaluate the objective
loss.backward()                        # 4. differentiate the objective
optimizer.step()                       # 5. update registered parameters
print('pre-update loss:', loss.item())
with torch.no_grad():
    print('post-update loss:', ((model(inputs)-targets)**2).mean().item())

Output

pre-update loss: 39.49003219604492
post-update loss: 7.869649410247803

The loss tensor still contains the pre-update loss; call the model again to evaluate new parameters. with establishes a context for the indented block. torch.no_grad() disables graph recording inside that block; it does not freeze parameters against all possible mutation.

Now repeat the same operations. This deliberately uses only three training observations to explain optimization; it does not estimate generalization performance.

Code

for epoch in range(60):
    optimizer.zero_grad()
    loss = ((model(inputs) - targets) ** 2).mean()
    loss.backward()
    optimizer.step()
print('trained predictions:', model(inputs).detach().squeeze(-1).numpy().round(2))

Output

trained predictions: [2.15 4.03 5.91]

.squeeze(-1) removes a final axis only if its length is one. Prefer an explicit axis: a bare .squeeze() can also remove a batch axis when a batch has one observation.

model.train() and model.eval() change the behavior of modules such as dropout and batch normalization. They do not turn gradient tracking on and off. For validation we normally use both model.eval() and torch.no_grad(). Our simple linear module has neither dropout nor batch normalization, but the distinction matters later.

Code

import copy
checkpoint = copy.deepcopy(model.state_dict())
model.eval()
with torch.no_grad():
    validation_example = model(torch.tensor([[4.]]))
print('new input prediction:', validation_example.item())
model.load_state_dict(checkpoint)

Output

new input prediction: 7.790169715881348

<All keys matched successfully>

state_dict() holds registered parameters and buffers. deepcopy() makes independent tensor copies. Chapter 2 saves such a snapshot only when validation loss improves. The test observations must remain outside that selection process.

A.8 · Follow the transformer dimensions#

For attention, axes represent batch, token position, head, and head coordinate. reshape changes how coordinates are grouped; transpose changes the order of axes. Neither operation learns weights.

Code

B, T, D, H = 2, 3, 8, 2
features = torch.arange(B*T*D).reshape(B, T, D).float()
by_head = features.reshape(B, T, H, D//H).transpose(1, 2)
restored = by_head.transpose(1, 2).contiguous().reshape(B, T, D)
print('before:', features.shape, 'heads:', by_head.shape, 'after:', restored.shape)
assert torch.equal(features, restored)

Output

before: torch.Size([2, 3, 8]) heads: torch.Size([2, 2, 3, 4]) after: torch.Size([2, 3, 8])

Start with (B,T,D). Split the last axis to obtain (B,T,H,D/H). Swap axes 1 and 2 to obtain (B,H,T,D/H). Each head now has its own token matrix. .contiguous() creates a conventional memory layout when needed; it does not alter the mathematical values. reshape can copy when required, while some uses of view require a compatible layout.

In Chapter 5, q, k, v = ...chunk(3, dim=-1) splits three equal feature blocks. This list comprehension:

Code

q, k, v = [a.reshape(B, T, H, D//H).transpose(1, 2) for a in (q, k, v)]

applies the same operation separately to three tensors. It is equivalent to writing three explicit assignments. For shapes (B,H,T,d), q @ k.transpose(-2,-1) has shape (B,H,T,T); softmax over dim=-1 normalizes each query’s weights over keys.

Code

scores = torch.zeros(3, 3)
allowed = torch.ones(3, 3, dtype=torch.bool).tril()
weights = torch.softmax(scores.masked_fill(~allowed, -torch.inf), dim=-1)
print(weights)
assert torch.allclose(weights.sum(-1), torch.ones(3))
assert torch.equal(weights.triu(1), torch.zeros(3, 3))

Output

tensor([[1.0000, 0.0000, 0.0000],
        [0.5000, 0.5000, 0.0000],
        [0.3333, 0.3333, 0.3333]])

tril() retains the lower triangle, including the diagonal. ~allowed negates the Boolean mask. Mask scores before softmax so disallowed keys receive zero probability after normalization. At least one allowed key is required per row.

A.9 · Structured requests, validation, and errors#

JSON is an interchange format for objects such as dictionaries, lists, strings, numbers, booleans, and null. Parsing JSON only checks syntax. A valid JSON object can still request an unsupported operation.

Code

import json
from dataclasses import dataclass

@dataclass(frozen=True)
class SummaryRequest:
    dataset: str
    statistic: str

def parse_request(text):
    payload = json.loads(text)
    if not isinstance(payload, dict) or set(payload) != {'dataset', 'statistic'}:
        raise ValueError('Expected exactly dataset and statistic')
    if not isinstance(payload['dataset'], str):
        raise ValueError('Dataset must be text')
    if payload['statistic'] not in ('mean', 'median'):
        raise ValueError('Unsupported statistic')
    return SummaryRequest(**payload)

request = parse_request('{"dataset": "pilot_a", "statistic": "mean"}')
print(request, request.statistic)

Output

SummaryRequest(dataset='pilot_a', statistic='mean') mean

A decorator such as @dataclass transforms the class definition; here it supplies an initializer and a readable representation. frozen=True prevents ordinary reassignment of its fields. **payload unpacks dictionary entries into named arguments. Type annotations document intent; they do not validate values automatically. The explicit checks still matter.

Code

try:
    parse_request('{"dataset": "pilot_a", "statistic": "delete"}')
except ValueError as error:
    print('Rejected:', error)

Output

Rejected: Unsupported statistic

try attempts an operation; except handles a specified failure. Do not catch every error and substitute a successful-looking answer. Preserve status, error type, and trace. In the LLM lab, text proposals never become arbitrary Python through eval() or exec().

A.10 · Exercises: predict, implement, investigate#

Predict. What are the shapes of X[0], X[:1], and X[:, :1]? Predict before printing.
Implement. Complete column_center below without a Python loop. Check that it preserves shape and produces zero column means.
Debug. Deliberately remove zero_grad() from a copy of the training loop. Explain the changed update in terms of accumulated derivatives; do not claim a general optimizer ranking from this example.
Trace. For (B,T,D,H)=(2,5,12,3), write every shape from features to attention scores and back.
Validate. Extend parse_request with a dataset allowlist. Test malformed JSON, extra keys, an unknown dataset, and a supported request.

Code

def column_center(array):
    # Replace ... with a vectorized expression.
    return ...

# Run after completing the function:
# centered_X = column_center(X)
# assert centered_X.shape == X.shape
# assert np.allclose(centered_X.mean(axis=0), 0)

Use the official Python tutorial, NumPy broadcasting guide, and PyTorch autograd introduction as references. Learn these operations through small computations before adding another framework.

A.11 · Files, local modules, and a fresh notebook kernel#

Read this before the first live notebook in C.1, and before any downloaded student assignment.

A module is a Python file containing reusable definitions. In the live labs, from labs.contracts import validate_response imports the same function used by Chapter 10. This avoids two validators quietly implementing different scientific inclusion rules. A downloaded notebook that imports course modules must stay inside the cloned course folder, or point to that folder with GENAI_COURSE_ROOT.

Code

from pathlib import Path
import os
import sys


start = Path(os.environ.get('GENAI_COURSE_ROOT', Path.cwd())).resolve()
root = None
# Examine this folder, then each enclosing folder.
candidates = [start] + list(start.parents)
for candidate in candidates:
    if (candidate / 'labs/model.json').is_file():
        root = candidate
        break
if root is None:
    raise FileNotFoundError('Keep the notebook inside Teaching-GenAI, or set GENAI_COURSE_ROOT')
if str(root) not in sys.path:
    sys.path.insert(0, str(root))
course_root = root
print('Python version:', sys.version.split()[0])
print('Course root found:', (root / 'labs/model.json').is_file())
print('Example relative path:', Path('labs') / 'contracts.py')

Output

Python version: 3.12.14
Course root found: True
Example relative path: labs/contracts.py

Path.cwd() is the current working directory, not necessarily the notebook’s saved folder. start.parents lists enclosing directories; list(...) makes them a list, and + joins lists. The loop checks the current folder first and moves outward until it finds labs/model.json. break stops at the first match. None is the initial “not found” marker. This works from labs/notebooks/, docs/notebooks/, or the repository root; an arbitrary folder outside the clone needs an explicit environment setting. os.environ.get(name, default) reads an optional environment setting and uses the default if absent. root / 'labs' joins paths; it does not divide strings. sys.path.insert(0, str(root)), used in the live notebooks, lets Python locate the local labs package. It does not install packages or load a model.

A notebook kernel is a running Python process. Running a cell twice can append two results or make two model calls. Restart Kernel and Run All Cells checks that the notebook does not depend on objects created out of order. A saved output is evidence from an earlier run, not evidence that the current kernel has those objects.

Before studying, check sys.executable in your own notebook: it should point to the course environment. If import torch works in a terminal but fails in Jupyter, compare the interpreters before changing the lesson code. NameError usually indicates a cell-order problem; ModuleNotFoundError indicates an environment or import-path problem; a rejected JSON response is a model/interface outcome after setup has succeeded.

A.12 · Expand compact syntax before using it#

Read this before Chapter 2, where comprehensions and default arguments already appear. Return before Chapters 5–6 for tensor-specific examples.

A list comprehension is an ordinary loop with its output collection written compactly. Begin with the loop while learning; use the shorter expression after you can predict its result.

Code

values = [2, 4, 6]
squares = []
for value in values:
    squares.append(value ** 2)
compact = [value ** 2 for value in values]
assert squares == compact == [4, 16, 36]
print(squares)

def affine(x, slope=1, intercept=0):
    return slope * x + intercept

settings = {'slope': 2, 'intercept': 1}
assert affine(3, **settings) == affine(3, slope=2, intercept=1) == 7
print('Expanded keyword arguments:', affine(3, **settings))

Output

[4, 16, 36]
Expanded keyword arguments: 7

**settings unpacks dictionary keys into named arguments; it does not exponentiate a dictionary. In a function definition, **settings collects extra named arguments into a dictionary. Chapter 6 passes decoding options this way. Default argument expressions are evaluated when a function is defined: features=Z captures the current object, so changing a global name later does not change that default.

@torch.no_grad() above a function applies a wrapper so calls do not record gradients. It is the function-wide version of with torch.no_grad():. lambda x: x + 1 is a short function expression; when a function becomes difficult to explain in one sentence, use a named def instead. These conveniences change how code is written, not the model or estimand.

A.13 · Read a record without mistaking it for an answer#

A list of dictionaries is a small table. Each dictionary is one observation, and each key is a variable name. In the common evaluation table, None means not yet reviewed; it must not be converted to success or discarded. An agent trajectory is one task outcome even when it contains five model calls.

For a frozen observation use deepcopy, because saved = messages creates another name for the same list, and messages.copy() still shares nested dictionaries. The following tiny example explains the Agent logging fix without loading a model.

Code

from copy import deepcopy
messages_demo = [{'role': 'user', 'content': 'Original question'}]
frozen_demo = deepcopy(messages_demo)
messages_demo[0]['content'] = 'Later edit'
messages_demo.append({'role': 'assistant', 'content': 'Later answer'})
assert frozen_demo == [{'role': 'user', 'content': 'Original question'}]
print('Current / saved message counts:', len(messages_demo), len(frozen_demo))

Output

Current / saved message counts: 2 1

A simple workflow is input → proposal → validation → computation → evidence → review. A function boundary packages steps you have already inspected. In C.1, generate produces IDs, slicing removes the prompt, and decode produces text. In C.3, the raw JSON string becomes an action dictionary, the dispatcher changes a state dictionary, and an observation is appended to the next prompt. In C.4, an interpreted request is compared with a separately reviewed request card before any simulation. None of these structural checks proves that generated prose is scientifically correct.

A.14 · Expand successful tool use and the capstone gate#

The first failed Agent lookup in C.3 leaves its state unchanged. A successful lookup does change it. Here every proposal is an explicitly scripted teaching fixture, so this cell makes no LLM calls. state is a dictionary whose observed value is another dictionary and whose results value is a list. Read one mutation at a time.

Code

from labs.guided_systems import dispatch
state_walk = {'observed': {}, 'results': []}
lookup_action = {'tool': 'lookup', 'arguments': {'dataset': 'pilot_b'}}
lookup_observation = dispatch(lookup_action, state_walk)
print('Observed keys after lookup:', list(state_walk['observed']))
print('Observed values:', lookup_observation['values'])
summary_action = {'tool': 'summarize',
                  'arguments': {'dataset': 'pilot_b', 'statistic': 'median'}}
summary_observation = dispatch(summary_action, state_walk)
assert summary_observation['value'] == 1.5
assert len(state_walk['results']) == 1
print('Computed result:', state_walk['results'][0])

Output

Observed keys after lookup: ['pilot_b']
Observed values: [1.0, 1.0, 2.0, 9.0]
Computed result: {'dataset': 'pilot_b', 'statistic': 'median', 'value': 1.5}

The lookup stores values obtained from the registered dataset. The summarize action supplies only a dataset identifier and statistic, not a replacement data vector. Sorting [1, 1, 2, 9] gives middle values 1 and 2, hence median 1.5. The executor can now validate a finish proposal against this computed result. A note inside the dataset remains data, not authority to change the calculation.

For a capstone, separate five objects: the original question string, a proposed dictionary, a separately reviewed dictionary, numerical tool evidence, and a report string. The following scripted fixture exposes the operations packaged by execute_confirmed in C.4. In the real notebook, interpret_scenario(lm, question) supplies the proposal through a recorded model call. Do not confuse this fixture with another measured model success.

Code

from labs.guided_systems import validate_spec, scope_reasons, report_simulation
from labs.llm_lab import coverage_tool

reviewed = {'family': 'normal', 'n': 25, 'method': 'z_estimated_sd',
            'confidence': 0.95, 'repetitions': 500, 'dependence': 'iid'}
proposal_walk = dict(reviewed)  # Scripted proposal, not generated text.
validate_spec(proposal_walk)  # Representation and numeric limits.
validate_spec(reviewed)
reasons = scope_reasons(reviewed)  # Scope is a separate check.
if reasons:
    outcome_walk = {'status': 'abstained', 'reasons': reasons}
elif proposal_walk != reviewed:
    outcome_walk = {'status': 'intent_mismatch'}
else:
    evidence_walk = coverage_tool(proposal_walk['family'], proposal_walk['n'],
                                  repetitions=proposal_walk['repetitions'])
    evidence_walk.update(method=proposal_walk['method'],
                         confidence=proposal_walk['confidence'],
                         dependence=proposal_walk['dependence'])
    report_walk = report_simulation(evidence_walk)
    outcome_walk = {'status': 'completed', 'evidence': evidence_walk,
                    'report': report_walk}
print(outcome_walk['status'])
print(outcome_walk.get('report', outcome_walk.get('reasons')))

Output

completed
For iid normal samples with n=25, the approximate z interval's estimated coverage is 0.928 from 500 repetitions. The approximate 95% Monte Carlo interval for that coverage probability is [0.902, 0.948]. The interval procedure's nominal 95% label does not guarantee its actual coverage.

The equality compares all six keys and values, not only family and n. update adds declared method metadata to numerical evidence; it does not recalculate coverage. get retrieves an optional key without raising KeyError. The report template inserts evidence from the tool. This example starts with an already reviewed card; the complete helper additionally blocks missing confirmation. Change the card to Student-t, 99%, dependent sampling, or 2000 repetitions and predict which branch runs before executing it. A correct numeric result for the wrong request is still a failed task.

A · Python and PyTorch, one operation at a time

Contents

A · Python and PyTorch, one operation at a time#

A.1 · Names, values, and containers#

A.2 · Functions, indentation, and small loops#

A.3 · Arrays, shapes, and axes#

A.4 · Random generators and reproducible simulation#

A.5 · Tensors and one derivative#

A.6 · What a model class means#

A.7 · Perform one update, then build a loop#

A.8 · Follow the transformer dimensions#

A.9 · Structured requests, validation, and errors#

A.10 · Exercises: predict, implement, investigate#

A.11 · Files, local modules, and a fresh notebook kernel#

A.12 · Expand compact syntax before using it#

A.13 · Read a record without mistaking it for an answer#

A.14 · Expand successful tool use and the capstone gate#