supervised learning

Supervised learning is a machine learning paradigm in which a model learns from labeled examples, meaning inputs that come paired with the correct answer, so that it can predict the answer for new data it has never seen.

Each example pairs a set of features with a label that a human annotator or a recorded outcome supplies. During training, the model predicts a label, a loss function scores how far that prediction sits from the true one, and an optimizer such as gradient descent nudges the weights to shrink the gap. Held-out data then measures whether the fit generalizes or merely memorizes.

The label’s data type splits the field into two task families:

• Classification: The label is one of a discrete set of classes, such as spam or not spam.
• Regression: The label is a continuous value, such as tomorrow’s rainfall in millimeters.

The stepper below runs the training loop on a fixed set of labeled examples, one gradient-descent step at a time. The dashed segments are the gaps that the loss measures, and switching the label type swaps a continuous fit for a two-class one.

Algorithms span logistic regression, support vector machines, decision-tree ensembles, and deep neural networks, and libraries such as scikit-learn expose the classic algorithms behind one fit-and-predict interface.

In large language model work, the paradigm shows up as supervised fine-tuning, where a pretrained base model trains on curated prompt-and-response pairs written by labelers. The 2022 InstructGPT paper used that stage to teach GPT-3 to follow instructions before applying reinforcement learning from human feedback.

The labels are also the bottleneck. Annotation costs time and money, and any bias baked into them gets learned right alongside the signal.

Related Resources

Tutorial

Logistic Regression in Python

In this step-by-step tutorial, you'll get started with logistic regression in Python. Classification is one of the most important areas of machine learning, and logistic regression is one of its basic methods. You'll learn how to create, evaluate, and apply a model to make predictions.

intermediate algorithms data-science machine-learning

For additional information on related topics, take a look at the following resources:

• Linear Regression in Python (Tutorial)
• The k-Nearest Neighbors (kNN) Algorithm in Python (Tutorial)
• Split Your Dataset With scikit-learn's train_test_split() (Tutorial)
• Practical Text Classification With Python and Keras (Tutorial)
• Learn Text Classification With Python and Keras (Course)
• Splitting Datasets With scikit-learn and train_test_split() (Course)
• Starting With Linear Regression in Python (Course)
• Linear Regression in Python (Quiz)
• Using k-Nearest Neighbors (kNN) in Python (Course)
• Split Your Dataset With scikit-learn's train_test_split() (Quiz)