Vectors, gradients and probability: the three tools every model is built from.

3 units / 9 lessons / about 85 min of reading / 1 project

1.1

Vectors and matrices

How data becomes numbers, and the two operations that do almost all the work: the dot product and the matrix multiply.

  1. lesson 1Data as vectors9 min · A data point is a list of numbers. Its length, direction and similarity are geometry.
  2. lesson 2Matrices as machines10 min · A matrix transforms vectors. A dataset is a matrix, and one multiply scores every row.
  3. lesson 3Thinking in arrays9 min · Vectorise instead of looping. Axes, reductions and broadcasting.
  4. theoryTheory test6 questions, 80% to pass
  5. practicalFind the most similar flower5 hidden checks
1.2

Derivatives and gradients

How a model knows which way to change: slopes, gradients and the chain rule.

  1. lesson 1Slopes and derivatives8 min · A derivative says how fast the output changes when you nudge the input.
  2. lesson 2Gradients and gradient descent10 min · With many parameters, the gradient points uphill. Step the other way, repeatedly.
  3. lesson 3The chain rule9 min · Derivatives of functions inside functions, the engine of backpropagation.
  4. theoryTheory test6 questions, 80% to pass
  5. practicalNumerical gradients and a descent loop5 hidden checks
1.3

Probability and statistics

Uncertainty in numbers: distributions, averages, Bayes' theorem and maximum likelihood.

  1. lesson 1Random variables and distributions9 min · Bernoulli, categorical and Gaussian: the shapes uncertainty usually takes.
  2. lesson 2Expectation, variance and correlation9 min · Summarising uncertainty, how sure an average is, and why correlation isn't causation.
  3. lesson 3Bayes' theorem and maximum likelihood11 min · Updating beliefs with evidence, and choosing parameters that make the data most probable.
  4. theoryTheory test6 questions, 80% to pass
  5. practicalBayes and maximum likelihood5 hidden checks
module project

Fit a line by gradient descent

Predict an iris petal's width from its length, training a model from scratch with the tools from this module.

Preview the project →
  1. Load and look
  2. The model and the loss
  3. The gradient, checked
  4. Train it
  5. Check against the exact answer
  6. Break it, then fix it with scaling