What Is Parameter-Efficient Fine-Tuning and How Does It Work?

This technique saves on cost because it trains just a fraction of the model. It gives you a customized model for your needs without the memory cost of training all the weights.

Full fine-tuning involves updating all the weights and requires a complete copy of the model per task. In PEFT, the base is frozen and only a few parameters are trained, which results in massive reductions in memory and storage compared with full fine-tuning.

LoRA is an example of PEFT, and perhaps the most used one. PEFT is the more general family that also covers adapter modules and prompt or prefix tuning, where only a limited number of parameters are trained.

Rarely more than slightly. For most tasks PEFT approximates full fine-tuning rather accurately. For tasks where the model needs to change fundamentally, full fine-tuning can still have an edge.

QLoRA is a combination of LoRA and a quantized base model, so the frozen weights require far less memory. QLoRA lets us fine-tune very large models even on limited hardware, while keeping LoRA's efficiency.