Data as vectors

A machine learning model never sees a flower, a photo or a sentence. It sees numbers. The first step of every project is turning each example into a fixed-length list of numbers called a feature vector. An iris flower becomes four measurements in centimetres: sepal length, sepal width, petal length, petal width. A 28×28 greyscale image becomes 784 pixel brightnesses. A word becomes a few hundred learned numbers (you'll meet those embeddings in module 5).

Writing $\mathbf{x}\in {\mathbb{R}}^4$ says "x is a list of 4 real numbers". Once data is a vector, it is also a point in space, or an arrow from the origin to that point. That is the whole trick: questions about data ("are these two flowers alike?") become questions about geometry ("are these two points close?").

Adding and scaling

Vectors add element by element, and multiplying by a number (a scalar) stretches every element. Geometrically, adding puts arrows head to tail, and scaling stretches an arrow without turning it (a negative scalar flips it around).

Length: the norm

The length of a vector is its Euclidean norm, Pythagoras in any number of dimensions. The distance between two points is the norm of their difference, $\Vert \mathbf{a}-\mathbf{b}\Vert$. This is the distance k-nearest-neighbours uses in module 2.

The dot product

The single most important operation in machine learning is the dot product: multiply matching elements and add them up. A linear model's prediction is a dot product of weights and features. A neuron computes a dot product. Attention in a transformer is built from dot products.

The second form is the geometric meaning: the dot product is large and positive when two arrows point the same way, zero when they are at right angles (orthogonal), and negative when they point apart. Drag the arrows below and watch the sign.

Divide out the lengths and you get cosine similarity, a number from −1 to 1 that measures direction only. It is how search engines and recommendation systems compare documents or users whose vectors have very different sizes: a long document and a short one about the same topic point the same way even though one vector is much longer.

A dot product asks "how much do these two vectors agree?". Almost every model you will build is a stack of dot products with learned numbers in one of the two vectors.

In code

We use numpy, the array library underneath nearly all scientific Python (scikit-learn, PyTorch and JAX all speak its language). Everything runs in your browser: the first run downloads Python, which takes a few seconds.