Week 7 - Transformers Part 1

Understanding the Transformer

Course session

Explanatory Session Part 1

Self-attention and multihead attention

Hugging Face Introduction

Library and Walk-through of HuggingFace101

Explanatory Session Part 2

Transformer Encoder and Positional Encoding



Go through this excellent site explaining Transformers:

Do Chapter 1 and Chapter 2 of the HuggingFace NLP course

Go through the TransformerHW1



Look closer at the Pytorch module nn.Transformer (documentation) and go through a tutorial on how to use it for next token prediction.

Last updated