Week 7 - Transformers Part 1

Understanding the Transformer

Course session

Explanatory Session Part 1

Self-attention and multihead attention

Hugging Face Introduction

Library and Walk-through of HuggingFace101

Google Colaboratory

Explanatory Session Part 2

Transformer Encoder and Positional Encoding

To-do

😊

Go through this excellent site explaining Transformers:

The Illustrated Transformer

Do Chapter 1 and Chapter 2 of the HuggingFace NLP course

Introduction - Hugging Face NLP Coursehuggingface

Go through the TransformerHW1

Google Colaboratory

😊😊

Tutorial 6: Transformers and Multi-Head Attention — UvA DL Notebooks v1.1 documentation

😊😊😊

Look closer at the Pytorch module nn.Transformer (documentation) and go through a tutorial on how to use it for next token prediction.