# Week 3 - Introduction to Transformers

### This week you will...

* Master the basics of attention and transformers
* Familiarize with advanced models

### Slides

{% file src="<https://4020123021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MHobCAnoTQkN71lOgdv%2Fuploads%2FbkJ7JlIBgGn0WBHcMqoj%2F06_11_23_Introduction_to_transformers.pdf?alt=media&token=ad305807-2bd7-49d0-8986-1778ac49b10a>" %}

### Learning Resources

* [Stanford Lecture on transformers](https://www.youtube.com/watch?v=yqV_YfBBtK0\&list=PLoROMvodv4rOwvldxftJTmoR3kRcWkJBp\&index=7)
* [Rasa series](https://www.youtube.com/watch?v=yGTUuEx3GkA) on Attention

### Until next week you should...

* [x] create your project repo accroding to the instructions provided [here](https://opencampus.gitbook.io/opencampus-machine-learning-program/projects/requirements)
* [x] complete your literature review according to the instructions provide in the template repo [here](https://github.com/opencampus-sh/ml-project-template/blob/main/0_LiteratureReview/INSTRUCTIONS.md)
* [x] create a ChatGPT account if you don´t already have one
* [x] complete [week 1](https://www.coursera.org/learn/generative-ai-with-llms) of the LLM Coursera course (Audit Mode)