autoencoder

An autoencoder is a neural network that learns to reconstruct its own input. An encoder compresses the input into a compact code, and a decoder rebuilds the input from that code, so the network learns a useful representation without any labeled data.

Those codes occupy a latent space, the set of compressed representations that the encoder can produce. Keeping that space narrower than the input creates a bottleneck that forces the network to preserve only the structure that reconstruction depends on, and the gap between input and output supplies the loss that training minimizes. Resizing the bottleneck on a small autoencoder trained on 16x16 digit images shows that trade-off directly:

Denoising autoencoders corrupt the input and learn to restore the clean version, while sparse autoencoders limit how many units may activate at once. Variational autoencoders (VAEs), introduced by Kingma and Welling in 2013, encode each input as a probability distribution rather than a fixed vector, which makes them generative models that can sample new data.

Latent diffusion image generators run diffusion inside a pretrained autoencoder’s latent space rather than on raw pixels. Interpretability researchers train sparse autoencoders on large language model activations to pull apart the overlapping concepts that a single neuron encodes.

Related Resources

Tutorial

Python AI: How to Build a Neural Network & Make Predictions

In this step-by-step tutorial, you'll build a neural network from scratch as an introduction to the world of artificial intelligence (AI) in Python. You'll learn how to train your neural network and make accurate predictions based on a given dataset.

intermediate ai data-science machine-learning

For additional information on related topics, take a look at the following resources:

• Generative Adversarial Networks: Build Your First Models (Tutorial)
• Building a Neural Network & Making Predictions With Python AI (Course)
• PyTorch vs TensorFlow for Your Python Deep Learning Project (Tutorial)
• Python Deep Learning: PyTorch vs Tensorflow (Course)
• Python AI: How to Build a Neural Network & Make Predictions (Quiz)