This week you will...

• get all details about the structure and form of the course.

• get to know your fellow course participants.

• get a general introduction into the current use of transformer models and their basic idea.

Learning Resources

Until next week you should...

• watch video 1 (Self-Attention) and video 2 (Keys, Values, Queries) of the Rasa Series on Transformers & Attention.

• note at least one question on each of the videos of above.

• do chapter 1 of the Hugging Face course.

This week you will...

• insights on the benefits of the Hugging Face Datasets library

• discuss the literature review on project tasks

• get some ideas on how to visualize sequence data

Learning Resources

Until next class you should...

• complete chapter 6 (The Tokenizers Library) of the Hugging Face course

• look into the characteristics of you dataset and:

  • write down the specifics of how your data was collected

  • create filter variables to group your input data according to special characteristics

  • consider the following questions:

    • What are potential biases in your training data?

    • Are there outliers in the dataset?

    • Are the classes balanced? (If you deal with a classification task.)

This week you will...

• learn about the basic structure of the transformer models

• get to know what tokenization is about and why it is important.

Learning Resources

• To get an idea about the number of tokens a text or word is made off in the GPT models, you may want to checkout this site.

Until next week you should...

• complete chapter 3 (Fine-Tuning a Pretrained Model) of the Hugging Face course.

• meet with the other participants of the project you are interested in to get a more concrete idea of the project goal.

The conditions to be met in order to receive a Certificate of Achievement (and ECTS) are:

• Attendance to at least 80% of the classes (it is allowed to miss a maximum 2 classes)

• Delivery of a machine learning project based on the Hugging Face API and including a corresponding documentation.

Attendance:

If you attend via Zoom, please make sure to use your full name, which should be the same that you used to register at edu.opencampus.sh. Otherwise your attendance will not be recorded!

Projects:

Check the Projects section to learn more about the projects.

This week you will...

• get to know the mechanism underlying the self-attention approach.

• get to know the basics of prompt design and how to apply them in a playground.

• get to know examples for the application of transformer models.

Learning Resources

Additional resource explaining the Transformer model:

• Text: The Illustrated Transformer by Jay Alammar

• Video: Transformer Neural Networks - EXPLAINED!

Until next week you should...

• Watch video 3 (Multi Head Attention) and video 4 (Transformers) of the Rasa Series on Transformers & Attention.

• Note at least one question on the each of the videos!

• Complete chapter 2 of the Hugging Face course.

• Think about the project you want to do and/or get in contact with course mates about it.

This week you will...

• get insights on the benefits of the Hugging Face Tokenizers library

• discuss the characteristics of the data you use in your project

Learning Resources

Until next class you should...

• complete at least two sections from chapter 7 (Main NLP Tasks) of the Hugging Face course

• calibrate a baseline model

This week you will...

• learn how to fine-tune a Hugging Face model with your own data

Learning Resources

Until next week you should...

• complete chapter 4 (Sharing Models and Tokenizers) and chapter 5 (The Datasets Library) of the Hugging Face course

• do a literature review on the task of you project:

  • Search for transformer models applied to similar problems

  • Focus on the structure of the input and of the output

  • Are there pretrained models that you can use?

  • Which type of model is best suited? Do you need tokenization?

  • Do you need a type of embedding layer?

This week you will...

• get insights on the preprocessing of different NLP and sequence classification tasks.

• get an idea on plausible hyperparameters to fit transformer models for different tasks.

• learn about different metrics to evaluate NLP models.

Learning Resources

Until next class you should...

• prepare your final presentation as described in week 8 .

This week you will...

• present your project in the final presentation. :-)

The presentation should take about 15 minutes and include the following aspects that correspond to the content of a model card for the Hugging Face library:

• model description

• intended uses & limitations

• training data

• training procedure

• variable and Metrics

• evaluation results

Check this section on Building a model card for more details.

Additionally, please also include into your presentation any things that you tried out and that didn't work, so we can all learn from your experiences.