Theory test: Sequence models

6 questions. Pass with 5 correct.

1. 01 Looking up token $i$ in an embedding matrix $E$ is equivalent to:

   Adding $i$ to every entry of $E$Multiplying the one-hot vector for $i$ by $E$, which selects row $i$Taking the mean of $E$Computing $E^{\top }E$
2. 02 An RNN shares its weights across:

   Different training runsTime stepsLayers onlyNothing; each step has its own weights
3. 03 A gradient is multiplied by about 0.9 at each of 50 time steps. What fraction remains?

   About 45%About 0.5%About 90%About 10%
4. 04 In an LSTM, a forget gate close to 1 means:

   The cell state is erasedThe cell state is kept almost unchangedThe output is suppressedThe input is ignored completely
5. 05 Gradient clipping is a remedy for:

   Vanishing gradientsExploding gradientsOverfittingSlow data loading
6. 06 What was the bottleneck in the original encoder-decoder RNN for translation?

   The vocabulary was too smallThe whole input sentence had to be compressed into one fixed-size vectorIt couldn't use GPUsIt had no embeddings