generative pre-trained transformer (GPT)
Generative pre-trained transformer (GPT) is a family of autoregressive large language models (LLMs) that use the transformer architecture and are pre-trained on large text corpora using a generative next-token objective. Recent GPT models extend the same next-token objective beyond text to multimodal data such as images and audio. Then, they’re adapted to tasks through prompting, fine-tuning, intermediate methods such as instruction tuning, or preference alignment with reinforcement learning from human feedback (RLHF). Current GPT models are also post-trained to reason: they spend internal reasoning tokens before answering, and you can tune how much of that thinking they do at inference time with a reasoning effort setting.
Most GPT models use a decoder-only stack of the transformer with masked self-attention and causal decoding, which enables efficient parallelization during training and fluent text generation across a wide range of downstream tasks. The newest flagships vary that stack. GPT-6 Astra, for example, uses a recurrent-depth or “looped transformer” design that reuses layers to spend more computation per token at inference time.
Related Resources
Course
Leverage OpenAI's API in Your Python Projects
Learn how to use the ChatGPT API with Python's openai library to send prompts, control AI behavior with roles, and get structured outputs.
For additional information on related topics, take a look at the following resources:
By Leodanis Pozo Ramos • Updated Sept. 7, 2026
