Back to Course
LLM Engineering: Transformers & RAG
Module 5 of 12
5. Fine-Tuning (LoRA)
1. PEFT (Parameter Efficient Fine-Tuning)
Training a 70B parameter model is impossible on consumer hardware. LoRA freezes the model and trains a tiny adapter layer (0.1% of parameters).
2. Unsloth
Use optimized libraries to train Llama-3 in minutes on Colab.