A linear model can only combine the features you give it. Much of practical machine learning, especially before deep learning, is feature engineering: transforming raw inputs so that a simple model can do well. Three transformations cover most cases.
Scaling
Suppose one feature is a house's area in square metres (around 100) and another is its number of floors (around 2). The loss surface is then stretched: tiny changes to the area weight move the loss a lot, while the floors weight barely matters. You saw this in the module 1 project: an elongated bowl forces a small learning rate, and progress along the shallow direction crawls.
Standardisation rescales each feature to mean 0 and standard deviation 1, , which rounds the bowl out. Alternatives are min-max scaling to and, for heavy-tailed quantities like incomes, taking a log first.
Compute and on the training set only, then apply those same numbers to validation and test data. Computing them on all the data leaks information about the test set into training. Module 3 is all about this kind of mistake.
Polynomial features: curves from a linear model
"Linear" in linear regression means linear in the weights, not in the inputs. Add as extra columns and the model fits curves, yet it is still solved by exactly the same normal equation or gradient descent.
Notice what happens at high degrees: the curve passes through every training point and swings wildly between them. Training error goes to zero while error on new data explodes. This is overfitting, the central problem of machine learning, and module 3 is devoted to it. Polynomial features are also why scaling matters even more: for is almost ten million.
Categories: one-hot encoding
A colour feature with values red, green and blue can't be coded as 1, 2, 3: that would claim blue is "three times" red and green sits between them. Instead create one binary column per category:
| colour | is_red | is_green | is_blue |
|---|---|---|---|
| red | 1 | 0 | 0 |
| green | 0 | 1 | 0 |
| blue | 0 | 0 | 1 |
With a bias term in the model, one of these columns is redundant (they always sum to 1, which is perfectly collinear with the ones column), so it's common to drop one category as the baseline. One-hot vectors come back in module 5 as the starting point for word embeddings.
Linear models are linear in their weights, not their inputs. Scale features so gradient descent behaves, expand them to capture curves, and one-hot encode categories.