reading list
Where this course comes from.
The lessons are distilled from these courses, books and papers. Every lesson also links the two or three best sources for its topic in its “go deeper” column.
Courses
University courses with free lectures and notes.
- CS229: Machine LearningStanford (Andrew Ng, Tengyu Ma et al.)The derivations behind modules 1–3: regression, classification, generalisation.
- CS231n: Deep Learning for Computer VisionStanfordBackprop and CNNs built from numpy. The model for modules 4 and 5.
- CS224n: Natural Language Processing with Deep LearningStanfordWord vectors, RNNs, attention, transformers and LLMs.
- 6.S191: Introduction to Deep LearningMITA fast, modern overview with labs, refreshed every year.
- Practical Deep Learning for Codersfast.ai (Jeremy Howard)Top-down and practical: train real models first, open them up later.
- Neural Networks: Zero to HeroAndrej KarpathyBuilds micrograd, makemore and a GPT from scratch. The spirit of module 6's project.
- Machine Learning SpecializationDeepLearning.AI and Stanford (Andrew Ng)The gentlest structured introduction, with anomaly detection and recommenders.
Books
Free online unless marked.
- Mathematics for Machine LearningDeisenroth, Faisal & OngLinear algebra, calculus and probability, aimed exactly at ML.
- An Introduction to Statistical LearningJames, Witten, Hastie & TibshiraniThe clearest book on classical ML and on evaluating models honestly.
- Dive into Deep LearningZhang, Lipton, Li & SmolaInteractive textbook with runnable code, from preliminaries to transformers.
- Deep LearningGoodfellow, Bengio & CourvilleThe reference text on the theory of deep networks.
- Understanding Deep LearningSimon J. D. PrinceModern and beautifully illustrated, including diffusion models and transformers.
- Pattern Recognition and Machine LearningChristopher BishopThe probabilistic view of ML, in depth.
Visual explanations
Pictures that make the maths click.
- Essence of Linear Algebra and Neural Networks3Blue1BrownAnimated intuition for vectors, gradients, backprop and transformers.
- Distilldistill.pubInteractive research articles, e.g. on momentum and feature visualisation.
- Christopher Olah's blogcolah.github.ioThe classic explanations of LSTMs and backpropagation.
- The Illustrated TransformerJay AlammarThe transformer, one diagram at a time.
- Seeing TheoryBrown UniversityProbability and statistics, visually.
Papers
The originals behind the ideas in modules 4 to 6.
- Learning representations by back-propagating errors (1986)Rumelhart, Hinton & WilliamsBackpropagation for neural networks.
- Gradient-based learning applied to document recognition (1998)LeCun, Bottou, Bengio & HaffnerLeNet and convolutional networks.
- ImageNet Classification with Deep CNNs (2012)Krizhevsky, Sutskever & HintonAlexNet, the start of the deep learning era.
- Adam: A Method for Stochastic Optimization (2014)Kingma & BaThe default optimiser.
- Deep Residual Learning for Image Recognition (2015)He, Zhang, Ren & SunResidual connections.
- Attention Is All You Need (2017)Vaswani et al.The transformer.
- Denoising Diffusion Probabilistic Models (2020)Ho, Jain & AbbeelDiffusion models.
- Training language models to follow instructions with human feedback (2022)Ouyang et al.InstructGPT and RLHF.