recurrent neural network (RNN)

A recurrent neural network (RNN) is a neural network architecture for sequential data. This network applies the same recurrent transformation at each time step and passes a hidden state forward, allowing information to persist over time.

Classical forms include vanilla RNNs and gated variants like LSTM and GRU, which mitigate the vanishing-gradient problem via gating mechanisms during backpropagation through time. Exploding gradients are usually handled separately by clipping the gradient norm.

Architectures can be unidirectional or bidirectional, and encoder-decoder setups, often augmented with attention, powered sequence-to-sequence tasks such as machine translation before transformers displaced them.

While transformer models now dominate many benchmarks, RNNs continue to be useful in streaming, low-latency, or resource-constrained environments. Recurrence has also returned in newer large-scale models: linear-recurrent and state-space architectures such as Mamba, RWKV, and xLSTM revisit the RNN idea to avoid attention’s quadratic cost on long contexts.

Related Resources

Course

Building a Neural Network & Making Predictions With Python AI

In this step-by-step course, you'll build a neural network from scratch as an introduction to the world of artificial intelligence (AI) in Python. You'll learn how to train your neural network and make accurate predictions based on a given dataset.

intermediate data-science machine-learning