fine-tuning

Fine-tuning is the process of adapting a pre-trained model to a new task or domain by continuing training on labeled, preference, or in-domain data, while starting from the model’s learned parameters.

In practice, fine-tuning ranges from updating all weights of a base model for a specific task to using parameter-efficient methods that keep most weights frozen and learn small additions, such as adapters or low-rank modules like LoRA. It also covers preference- and reward-based methods, such as direct preference optimization and reinforcement fine-tuning, which learn from ranked or graded responses instead of fixed target outputs, as in RLHF.

Fine-tuning typically yields strong performance with modest data and compute compared to training from scratch, but it must balance adaptation with risks like overfitting or forgetting prior capabilities.

Related Resources

Tutorial

Hugging Face Transformers: Leverage Open-Source AI in Python

As the AI boom continues, the Hugging Face platform stands out as the leading open-source model hub. In this tutorial, you'll get hands-on experience with Hugging Face and the Transformers library in Python.

intermediate ai

For additional information on related topics, take a look at the following resources:

• Hugging Face Transformers (Quiz)