Ian Goodfellow and colleagues proposed generative adversarial networks in 2014 with a game. A generator turns random noise into fake samples. A discriminator sees real and fake samples and outputs the probability that its input is real. They train in alternation: to tell real from fake, to fool .
The discriminator's objective is just binary cross-entropy (module 2) on real-versus-fake labels. In theory, with unlimited capacity, the game's equilibrium is a generator whose samples match the data distribution exactly, at which point outputs 0.5 everywhere. In practice the generator usually maximises instead of minimising , because the original form gives vanishing gradients early on when easily spots the fakes.
Hard to train, stunning when it works
- Mode collapse: the generator finds a few outputs that fool the discriminator and produces only those, ignoring the variety of the data.
- Instability: two networks chasing each other can oscillate instead of converging. Many tricks (careful architectures like DCGAN, Wasserstein losses, spectral normalisation) exist to stabilise them.
- No likelihood: GANs give samples but no probability for a given image, which makes them hard to evaluate. Metrics like FID compare statistics of generated and real images in a pretrained network's feature space.
At their best GANs produced the first truly photorealistic synthetic faces (StyleGAN, Karras et al., 2019). Since around 2021, diffusion models have largely replaced them for image generation because they train stably and cover the data's variety better, though GAN-style adversarial losses survive inside many systems, for example to sharpen fast samplers.
A GAN pits a generator against a discriminator trained with cross-entropy. The adversarial signal can produce sharp, realistic samples, but training is unstable and prone to mode collapse.