Week 2 - Tokens & Embeddings revisted

This week you will...

• Gain a broad understanding of how language models have evolved, from early methods like n-grams to advanced transformer architectures.

• Understand the significance and limitations of word embeddings and recurrent neural networks, including LSTMs.

Slides

851KB

30_10_23_Token_and_embeddings_revisited.pdf

pdf

Learning Resources

• Stanford CS224N NLP with Deep Learning

• Stanford XCS224U: NLU

Until next week you should...

• Watch the last ten minutes of this Stanford lecture on Attention

• Watch the first 25 Minutes of this Stanford lecture on Attention

• Watch all 4 Videos of the Rasa Attention Series

• Complete the Notebook and play around with different embeddings

  • The linked Repo contains the notebook as well as two exemplary vector stores to play around with

• Optional: Watch this Stanford lecture on Transformers