temperature

Temperature is a decoding parameter that rescales the model’s logits before softmax, thereby controlling how deterministic or diverse the output is.

Lowering temperature sharpens the distribution, pushing probability mass toward the top tokens and approaching greedy decoding in the limit. Raising temperature flattens the distribution, allowing more exploration and novelty at the cost of coherence or correctness. Even the low end doesn’t guarantee reproducibility, though. A temperature of 0.0 still won’t make results fully deterministic.

Temperature is implemented by dividing logits by the temperature constant before softmax, and it is often used together with top-k or nucleus (top-p) sampling. In practice, one chooses temperature per task: low for factual or precise generation, and higher for creative or open-ended generation. Not every model exposes the parameter. Some current reasoning models reject non-default temperature values with an error, so prompting or a reasoning-effort setting steers their output instead.