# Week 2 - Tokens & Embeddings revisted

### This week you will...

* Gain a broad understanding of how language models have evolved, from early methods like n-grams to advanced transformer architectures.
* Understand the significance and limitations of word embeddings and recurrent neural networks, including LSTMs.

### Slides

{% file src="https://4020123021-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MHobCAnoTQkN71lOgdv%2Fuploads%2FeTkduZAde6qowlmABTyl%2F30_10_23_Token_and_embeddings_revisited.pdf?alt=media&token=7014f632-3137-4f0a-89ea-25b93b72f353" %}

### Learning Resources

* [Stanford CS224N](https://www.youtube.com/watch?v=rmVRLeJRkl4\&list=PLoROMvodv4rOSH4v6133s9LFPRHjEmbmJ) NLP with Deep Learning
* [Stanford XCS224U](https://www.youtube.com/watch?v=K_Dh0Sxujuc\&list=PLoROMvodv4rOwvldxftJTmoR3kRcWkJBp): NLU

### Until next week you should...

* [x] Watch the last ten minutes of this [Stanford lecture](https://www.youtube.com/watch?v=wzfWHP6SXxY\&t=4366s) on Attention
* [x] Watch the first 25 Minutes of this [Stanford lecture](https://www.youtube.com/watch?v=gKD7jPAdbpE\&list=PLoROMvodv4rOSH4v6133s9LFPRHjEmbmJ\&index=8) on Attention
* [x] Watch all 4 Videos of the [Rasa Attention Series](https://www.youtube.com/watch?v=yGTUuEx3GkA\&t=2s)
* [x] Complete the [Notebook](https://github.com/JP-CAU/Deep-Dive-LLMs/tree/main/Notebooks) and play around with different embeddings
  * [x] The linked Repo contains the notebook as well as two exemplary vector stores to play around with
* [x] Optional: Watch this [Stanford lecture](https://www.youtube.com/watch?v=ptuGllU5SQQ\&list=PLoROMvodv4rOSH4v6133s9LFPRHjEmbmJ\&index=9) on Transformers
