Machine learning code looks different from ordinary code: there are almost no loops over examples. Instead, whole arrays are combined at once. This is called vectorisation, and it is not just style. A Python for loop handles one number at a time through the interpreter; a numpy operation hands the whole array to compiled C code that uses SIMD instructions and cache-friendly memory access. The difference is commonly 10 to 100 times, and on a GPU much more.
Shape is everything
Every array has a shape, a tuple of its sizes along each axis. A vector of 4 numbers has shape (4,). The iris data has shape (150, 4). A batch of 32 colour images of 224×224 pixels has shape (32, 224, 224, 3). When code breaks, print shapes first.
Reductions along an axis
Functions like sum, mean, max and argmax can collapse one axis. X.mean(axis=0) collapses the rows and leaves one mean per column (per feature). X.mean(axis=1) collapses the columns and leaves one mean per row (per example). A useful way to remember it: the axis you name is the one that disappears.
Broadcasting
What should X - mu do when X is (150, 4) and mu is (4,)? Numpy broadcasts: it stretches the smaller array across the missing dimension without copying memory, so each row of X gets the same mu subtracted. The rule is to line the shapes up from the right; two sizes are compatible when they are equal or one of them is 1 (a missing leading dimension counts as 1).
| shapes | result | why |
|---|---|---|
(150, 4) and (4,) | (150, 4) | trailing 4 = 4; the vector repeats down the rows |
(150, 4) and (150, 1) | (150, 4) | 1 stretches to 4; the column repeats across |
(150, 4) and (150,) | error | trailing 4 ≠ 150 |
(3, 1) and (1, 5) | (3, 5) | both stretch: an outer sum |
The third row is the classic bug: X - X.mean(axis=1) fails because the per-row means have shape (150,). Pass keepdims=True to keep the collapsed axis as size 1, giving (150, 1), and it broadcasts correctly.
Standardising a dataset in one line
Many models train better when every feature has mean 0 and standard deviation 1 (you'll see why in module 2). With broadcasting this is one line, and it reads exactly like the formula :
- Boolean masks:
X[y == 0]keeps the rows where the condition is true. No loop, noif. - Fancy indexing:
X[[0, 5, 9]]picks rows by index;np.argsortgives the order that sorts. - Views, not copies: slicing like
X[:, 2]shares memory withX. Modify it and you modifyX. Use.copy()when you need independence.
Write the formula, not the loop. If you are writing for i in range(len(X)), there is usually a one-line array expression that is faster and closer to the maths.