softmax

Softmax is a function that converts a list of raw scores into a probability distribution, where every value falls between zero and one and all the values add up to one. In neural networks, those raw scores are called logits.

The function exponentiates each score and divides it by the sum of all the exponentials. Exponentiation makes every output positive and keeps the original ranking, while widening the gaps so that the highest score claims most of the probability. Hence the name: softmax is a smooth, differentiable version of picking only the highest score, so gradient descent can train a model through it.

In a large language model, softmax appears in two places. The output layer applies it to one logit per token in the vocabulary to produce next-token probabilities, and temperature reshapes that distribution by dividing the logits first. Inside each attention layer, softmax turns similarity scores into the weights that decide how much each token draws from the others.

The widget below applies softmax to five candidate next tokens. Dragging a logit or the temperature slider shows how the probabilities shift while their sum stays at one.

Interactive diagram — enable JavaScript to view.

During training, the output probabilities feed into cross entropy loss. Because exponentials overflow quickly, implementations subtract the largest score from every score before exponentiating, which leaves the result unchanged. With only two options, softmax reduces to the sigmoid activation function.

Python AI: How to Build a Neural Network & Make Predictions

Tutorial

Python AI: How to Build a Neural Network & Make Predictions

In this step-by-step tutorial, you'll build a neural network from scratch as an introduction to the world of artificial intelligence (AI) in Python. You'll learn how to train your neural network and make accurate predictions based on a given dataset.

intermediate ai data-science machine-learning

For additional information on related topics, take a look at the following resources:

Have a question about this? Mentor AI can show you examples, compare related terms, and point you to tutorials.

  • Ask Mentor AI
  • Pick a question and Mentor AI takes it from here.
By Martin Breuss • Updated Nov. 3, 2026
AI Coding Glossary Share Feedback