training
Training is the process of fitting a model’s parameters to data by minimizing or optimizing a carefully chosen objective (loss or surrogate) via gradient-based (or other) optimization, typically using forward and backward passes.
In practice, training workflows include:
- Specifying the objective (loss) and evaluation metrics
- Splitting the data into training, validation, and test sets
- Iterating over mini-batches across multiple epochs or streaming passes
- Updating weights with optimizers
- Applying regularization techniques, such as L1/L2 weight decay, dropout, normalization, data augmentation, early stopping, checkpointing, gradient clipping, and learning rate scheduling
- Optionally staging the run as pretraining followed by post-training, which covers supervised fine-tuning, preference optimization, and RLHF-style alignment in a pipeline
- Enhancing efficiency and stability through batching, normalization, mixed precision arithmetic, gradient accumulation, distributed/parallel training, and use of accelerated hardware (GPUs, TPUs)
Modern training often operates in overparameterized regimes, where implicit biases of optimization dynamics and early regularization can influence generalization.
Related Resources
Tutorial
Split Your Dataset With scikit-learn's train_test_split()
In this tutorial, you'll learn why splitting your dataset in supervised machine learning is important and how to do it with train_test_split() from scikit-learn.
For additional information on related topics, take a look at the following resources:
By Leodanis Pozo Ramos • Updated Aug. 9, 2026
