generative pre-trained transformer (GPT)

Generative pre-trained transformer (GPT) is a family of autoregressive large language models (LLMs) that use the transformer architecture and are pre-trained on large text corpora using a generative next-token objective. Then, they’re adapted to tasks through prompting, fine tuning, or intermediate methods such as instruction tuning.

GPT models leverage the transformer’s self-attention mechanism and causal decoding, enabling efficient parallelization during training and fluent text generation across a wide range of downstream tasks.