The conditions to be met in order to receive a Certificate of Achievement (and ECTS) are:

• Attendance to at least 80% of the classes (it is allowed to miss a maximum 2 classes)

• GitHub repository including a well-documented project on a fine-tuned large language model

• Presentation of the project at the end of the course

Attendance:

If you attend via Zoom, please make sure to use your full name, which should be the same that you used to register at edu.opencampus.sh. Otherwise your attendance will not be recorded!

Projects:

Check the Projects section to learn more about how to get started, complete, and submit you project.

Before the first class you should ...

• If you are not familiar with the Hugging Face API yet, do at minimum chapter 1 and 2 of the Hugging Face NLP course before the start of the course. Further, I strongly recommend to additionally complete the chapters 3, 5 and 6.

• Register to the Alpha Signal newsletter.

• Get a Discord account and enter the LAION community and the EleutherAI community.

This week you will...

• get all details about the structure and form of the course.

• get to know your fellow course participants.

• discuss the general types of training large language models.

• discuss current sources for staying up-to-date.

Learning Resources

• Hugging Face NLP course

• YouTube channel from Yannic Kilcher

Until next week you should...

• decide on a project.

• watch Creating your own ChatGPT: Supervised fine-tuning (SFT) from Niels Rogge (1 hour).

• note at least one question on the video above.

• watch the four videos of the Rasa Algorithm Whiteboard on Transformers and Attention (Part 1 to Part 4; about 50 minutes in total) if the transformers architecture is new to you.

This week you will...

• discuss the definition of your project for the course.

• discuss issues of Niels Rogge's video on Fine-Tuning

Learning Resources

Until next week you should...

• answer the following two questions: - Which model do you want to fine-tune? - How do you want to evaluate it?

• do the short course (1 hour).

This week you will...

• learn about different ways of using special tokens for instruction tuning

• discuss problems in model evaluation

Learning Resources

• on how to fine-tune with Hugging Face AutoTrain

Until next week you should...

• continue working on your project and prepare the project presentation.

This week you will...

• present your project in the final presentations. :-)

After the Final Presentation you should...

• complete your project documentation and submit it according to the instructions given

This week you will...

• learn about different ways of using special tokens for instruction tuning

• discuss problems in model evaluation

Learning Resources

• Notebook showing different examples of tokenizations

Until next week you should...

• focus on implementing the evaluation chain if you do not have one yet

This week you will...

• discuss the definition of your project for the course.

• discuss issues of Niels Rogge's video on Fine-Tuning

Learning Resources

• Creating your own ChatGPT: Supervised fine-tuning (SFT) from Niels Rogge

Until next week you should...

• do a “literature review” for your project.

• watch the first two chapters of LLM Engineering: Structured Outputs (“Asking LLMs for Structured Data” and “Prompting LLMs”).

• note a least one question on the videos above.

This week you will...

• discuss important aspects of evaluating models.

• discuss which models you want to use for your project.

Learning Resources

Until next week you should...

• decide on a baseline model and implement an evaluation chain for your base model.