# Week 2 - Self-Attention and Prompt Design

### This week you will...

* get to know the mechanism underlying the self-attention approach.
* get to know the basics of prompt design and how to apply them in a playground.
* get to know examples for the application of transformer models.

### Learning Resources

{% file src="https://4020123021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MHobCAnoTQkN71lOgdv%2Fuploads%2F8eXRYVsRpzIQHDZBfx45%2F221102_Self-Attention%20and%20Prompt%20Design.pdf?alt=media&token=9ff45f61-8168-4814-9e07-ee4f2deaad40" %}

Additional resource explaining the Transformer model:

* Text: [The Illustrated Transformer](https://jalammar.github.io/illustrated-transformer/) by Jay Alammar
* Video: [Transformer Neural Networks - EXPLAINED!](https://www.youtube.com/watch?v=TQQlZhbC5ps)

### Until next week you should...

* [x] Watch [video 3](https://www.youtube.com/watch?v=23XUv0T9L5c\&list=PL75e0qA87dlG-za8eLI6t0_Pbxafk-cxb\&index=11) (Multi Head Attention) and [video 4](https://www.youtube.com/watch?v=EXNBy8G43MM\&list=PL75e0qA87dlG-za8eLI6t0_Pbxafk-cxb\&index=12) (Transformers) of the Rasa Series on Transformers & Attention.
* [x] Note at least one question on the each of the videos!
* [x] Complete [chapter 2](https://huggingface.co/course/chapter2/1) of the Hugging Face course.
* [x] Think about the project you want to do and/or get in contact with course mates about it.
