Week 7 - Transformers & Hugging Face

Course session

Explanatory Session Part 1

Self-attention and multihead attention

Hugging Face Introduction

Library and Use of HuggingFace for working with transfomer models

Explanatory Session Part 2

Transformer Encoder and Positional Encoding

Explanatory Session Part 3

Vision Transformer


Finetuning Vision Transformer on Kaggle Paddy Dataset



Look at current Kaggle competitions and make proposals


Go through this excellent site explaining Transformers:

Do Chapter 1-3 of the HuggingFace NLP course


Look closer at the Pytorch module nn.Transformer (documentation) and go through a tutorial on how to use it for next token prediction.

Watch this excellent "Build from Scratch" video from Andrej Karpathy

Last updated