Theory test: Linear regression

6 questions. Pass with 5 correct.

1. 01 For $L(\mathbf{w})=\frac{1}{n}\Vert X\mathbf{w}-\mathbf{y}{\Vert }^2$, what is ${\nabla }_{\mathbf{w}}L$?

   $\frac{2}{n}{X}^{\top }(X\mathbf{w}-\mathbf{y})$$\frac{2}{n}X(X\mathbf{w}-\mathbf{y})$$\frac{2}{n}(X\mathbf{w}-\mathbf{y})$$X^{\top }X\mathbf{w}$
2. 02 Why does standardising features usually speed up gradient descent?

   It reduces the number of featuresIt makes the loss surface rounder, so one learning rate suits every directionIt makes the model non-linearIt removes outliers
3. 03 You fit $\hat{y}=w_{1}x+w_2{x}^2+w_3{x}^3+b$ with the normal equation. This is possible because:

   The normal equation works for any modelThe model is linear in its weights, even though it's a curve in $x$Cubic functions are convexIt isn't possible; you need gradient descent
4. 04 What is the main practical limitation of solving the normal equation directly?

   It only finds a local minimumIt needs a well-chosen learning rateSolving the $(d\times d)$ system grows like $d^3$, and fails when $X^{\top }X$ is singularIt can't include a bias term
5. 05 A model predicts price with weight $+12,000$ on "bedrooms". The correct interpretation is:

   Bedrooms cause prices to rise by 12,000 eachOne more bedroom raises the prediction by 12,000, holding the other features fixedHouses have on average 12,000 bedroomsBedrooms are the most important feature
6. 06 What does a step of mini-batch SGD use?

   The gradient on the whole training setThe gradient on a small random subset of examplesThe gradient on the test setA random direction