A multilayer perceptron (MLP, or fully connected network) chains layers. Each layer multiplies by a weight matrix, adds a bias, and applies an activation. The last layer usually has no activation and outputs logits, which a softmax turns into class probabilities. Despite the name, it uses smooth activations, not perceptron steps.
The forward pass, in batches
With a batch of examples stacked as rows of , the whole network is a few matrix expressions. For one hidden layer:
The bias vectors broadcast across the batch. Writing networks this way is not just tidy: matrix multiplies are what GPUs are built to do fast, so a network's speed is largely its count of multiply-adds.
Counting parameters
A dense layer from to units has weights plus biases. A classic MNIST network, 784 → 128 → 10, has parameters. The first layer dominates, which is one reason images need a more economical design (convolutions, module 5).
A network learns its own features
Here is the conceptual leap. In module 2 you engineered features by hand (polynomials, one-hot encodings) and then fitted a linear model. An MLP's last layer is a linear model (softmax regression), and the layers before it are a feature extractor whose features are learned by gradient descent along with the classifier. The hidden layer of a digit classifier ends up detecting strokes and curves nobody programmed. This is called representation learning, and it's why deep learning took over perception tasks where hand-made features had plateaued.
Depth versus width: wider layers add capacity at one level of abstraction, deeper stacks compose features of features. In practice moderately deep networks generalise better per parameter, but very deep plain MLPs are hard to train; the next two units explain why and what fixes it.
An MLP alternates matrix multiplies and non-linearities. Its hidden layers learn features; its last layer is a linear classifier on top of them.