Name		Name	Last commit message	Last commit date
parent directory ..
.DS_Store		.DS_Store
Fine_tune_a_Mistral_7b_model_with_DPO.ipynb		Fine_tune_a_Mistral_7b_model_with_DPO.ipynb
LLM-Course Lecture 4.pdf		LLM-Course Lecture 4.pdf
README.md		README.md
supervised_finetuning.ipynb		supervised_finetuning.ipynb

README.md

Fine-tuning LLMs

Lecture slides: LLM-Course Lecture 4

Lab Exercise

Run the supervised_finetuning.ipynb in Google Colab or CSC (convert to python script)

Change the base model used (search for small <7B parameter models in Hugging Face).
Change the dataset used in fine-tuning.

Bonus challenge:
- Change the fine-tuning method from supervised fine-tuning to DPO.
- Change the code accordingly, see: Hugging Face DPO Trainer Documentation
- Select an appropriate DPO dataset. Search Hugging Face Datasets.