Theory test: Neighbours, trees and ensembles

6 questions. Pass with 5 correct.

1. 01 What is the Gini impurity of a node with two classes split 50/50?

   00.250.51
2. 02 A 1-nearest-neighbour classifier, evaluated on its own training set (no duplicate points), gets what accuracy?

   About 50%100%The same as on the test setIt depends on the number of features
3. 03 How does a random forest make its trees less correlated with each other?

   By training each tree on the test setBootstrap samples, plus a random subset of features considered at each splitBy making every tree depth 1By training trees in sequence on residuals
4. 04 In gradient boosting, each new model is fitted to:

   A fresh bootstrap sample of the targetsThe residuals (negative gradient of the loss) of the ensemble so farThe test setRandom noise
5. 05 You run k-NN on features "income in dollars" and "age in years" without scaling. What happens?

   Nothing; k-NN is scale-invariantIncome dominates the distance and age is mostly ignoredAge dominates the distanceThe algorithm fails to run
6. 06 Averaging $B$ models with correlation $\rho =0.5$: as $B\to \infty$, the variance of the average approaches:

   0$0.5{\sigma }^2$${\sigma }^2$${\sigma }^2/B$