Week 7 - Transformers & Hugging Face

Course session

Explanatory Session Part 1

Self-attention and multihead attention

Hugging Face Introduction

Library and Use of HuggingFace for working with transfomer models

Explanatory Session Part 2

Transformer Encoder and Positional Encoding

Explanatory Session Part 3

Vision Transformer

Walk-through

Finetuning Vision Transformer on Kaggle Paddy Dataset

To-do

😊

Look at current Kaggle competitions and make proposals

😊😊

Go through this excellent site explaining Transformers:

Do Chapter 1-3 of the HuggingFace NLP course

😊😊😊

Look closer at the Pytorch module nn.Transformer () and go through a on how to use it for next token prediction.

Watch this excellent "Build from Scratch" video from Andrej Karpathy