fine-tuning
Fine-tuning is the process of adapting a pre-trained model to a new task or domain by continuing training on labeled, preference, or in-domain data, while starting from the model’s learned parameters.
In practice, fine-tuning ranges from updating all weights of a base model for a specific task to using parameter-efficient methods that keep most weights frozen and learn small additions, such as adapters or low-rank modules like LoRA.
Fine-tuning also covers preference- and reward-based methods that learn from ranked or graded responses instead of the fixed target outputs used in supervised fine-tuning (SFT). Examples include direct preference optimization (DPO) and reinforcement fine-tuning (RFT). Some of these methods, like RLHF, first fit a separate reward model, while DPO optimizes the preference data directly and skips that step.
Fine-tuning typically yields strong performance with modest data and compute compared to training from scratch, but it must balance adaptation with risks like overfitting or forgetting prior capabilities.
Related Resources
Tutorial
Hugging Face Transformers: Leverage Open-Source AI in Python
Get hands-on with Hugging Face Transformers in Python. Download, run, and inspect pretrained open-source AI models, and speed up inference on GPUs.
For additional information on related topics, take a look at the following resources:
Have a question about this? Mentor AI can show you examples, compare related terms, and point you to tutorials.
Ask Mentor AI
By Leodanis Pozo Ramos • Updated Sept. 21, 2026
