# Week 7 - Transformers Part 1

## Course session

**Explanatory Session Part 1**

Self-attention and multihead attention

**Hugging Face Introduction**

Library and Walk-through of HuggingFace101

{% embed url="<https://colab.research.google.com/drive/1r74kJw-OvukVnR4a1QTxMp8qDgf69Pt9?usp=sharing>" %}

**Explanatory Session Part 2**

Transformer Encoder and Positional Encoding

## To-do

😊

Go through this excellent site explaining Transformers:&#x20;

{% embed url="<http://jalammar.github.io/illustrated-transformer/>" %}

Do Chapter 1 and Chapter 2 of the HuggingFace NLP course

{% embed url="<https://huggingface.co/learn/nlp-course/chapter1/1>" %}

Go through the TransformerHW1

{% embed url="<https://colab.research.google.com/drive/190nI2yzsl8sm25kM2sKhdVNj-4Yf3d5N?usp=sharing>" %}

😊😊

{% embed url="<https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial6/Transformers_and_MHAttention.html>" %}

😊😊😊

Look closer at the Pytorch module `nn.Transformer` ([documentation](https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html)) and go through a [tutorial](https://pytorch.org/tutorials/beginner/transformer_tutorial.html) on how to use it for next token prediction.