4 hidden checks
Implement embedding lookup and a vanilla RNN over a whole sequence.
embed(tokens, E): given integer token ids (a 1-D array) and an embedding matrixE(V, D), return the(T, D)array of their vectors.rnn_forward(X, h0, Wx, Wh, b):Xhas shape(T, D),h0shape(H,),Wx(D, H),Wh(H, H),b(H,). Apply for each step and return all hidden states stacked, shape(T, H).
- embed selects rowsnot run
- rnn_forward returns one state per stepnot run
- rnn_forward matches the recurrencenot run
- the state carries information forwardnot run