This week you will...

• get all details about the structure and form of the course.

• get to know your fellow course participants.

• get a general introduction into the current use of transformer models and their basic idea.

Learning Resources

Until next week you should...

• watch (Self-Attention) and (Keys, Values, Queries) of the Rasa Series on Transformers & Attention.

• note at least one question on each of the videos of above.

• do of the Hugging Face course.

The conditions to be met in order to receive a Certificate of Achievement (and ECTS) are:

• Attendance to at least 80% of the classes (it is allowed to miss a maximum 2 classes)

• Delivery of a machine learning project based on the Hugging Face API and including a corresponding documentation.

Attendance:

If you attend via Zoom, please make sure to use your full name, which should be the same that you used to register at edu.opencampus.sh. Otherwise your attendance will not be recorded!

Projects:

Check the Projects section to learn more about the projects.

This week you will...

• get to know the mechanism underlying the self-attention approach.

• get to know the basics of prompt design and how to apply them in a playground.

• get to know examples for the application of transformer models.

Learning Resources

Additional resource explaining the Transformer model:

• Text: by Jay Alammar

• Video:

Until next week you should...

• Watch (Multi Head Attention) and (Transformers) of the Rasa Series on Transformers & Attention.

• Note at least one question on the each of the videos!

• Complete of the Hugging Face course.

This week you will...

• insights on the benefits of the Hugging Face Datasets library

• discuss the literature review on project tasks

• get some ideas on how to visualize sequence data

Learning Resources

Until next class you should...

• complete (The Tokenizers Library) of the Hugging Face course

• look into the characteristics of you dataset and:

  • write down the specifics of how your data was collected

This week you will...

• learn about the basic structure of the transformer models

• get to know what tokenization is about and why it is important.

Learning Resources

• To get an idea about the number of tokens a text or word is made off in the GPT models, you may want to checkout .

Until next week you should...

• complete (Fine-Tuning a Pretrained Model) of the Hugging Face course.

• meet with the other participants of the project you are interested in to get a more concrete idea of the project goal.

This week you will...

• learn how to fine-tune a Hugging Face model with your own data

Learning Resources

Until next week you should...

• complete (Sharing Models and Tokenizers) and (The Datasets Library) of the Hugging Face course

• do a literature review on the task of you project:

  • Search for transformer models applied to similar problems

This week you will...

• get insights on the preprocessing of different NLP and sequence classification tasks.

• get an idea on plausible hyperparameters to fit transformer models for different tasks.

• learn about different metrics to evaluate NLP models.

Learning Resources

Until next class you should...

• prepare your final presentation as described in week 8 .

This week you will...

• present your project in the final presentation. :-)

The presentation should take about 15 minutes and include the following aspects that correspond to the content of a model card for the Hugging Face library:

• model description

• intended uses & limitations

• training data

Check this section on for more details.

Additionally, please also include into your presentation any things that you tried out and that didn't work, so we can all learn from your experiences.

This week you will...

• get insights on the benefits of the Hugging Face Tokenizers library

• discuss the characteristics of the data you use in your project

Learning Resources

Until next class you should...

• complete at least two sections from (Main NLP Tasks) of the Hugging Face course

• calibrate a baseline model