arrow-left

All pages
gitbookPowered by GitBook
1 of 15

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Hello and welcome😊

Great that you want to dive into the deep water in Machine Learning. These are exciting times with major advancements on a quarterly basis like ChatGPT, Whisper, StableDiffusion and so many more. Nevertheless all these exciting models were developed with solid ML knowledge which is what we want to acquire in this course.

This is a course which brings you from beginner to intermediate or even advanced. It is formally called Intermediate Machine Learning but following HuggingFaceπŸ€— terms which we will use heavily in the course I like to call the course SmilingFace😊. This is meant ironically, because you will never laugh in the course😊. Okay joking aside the use of smileys during learning and practioning ML helps us to remember to have fun, laugh about our mistakes and take ourself not seriously as it was proposed by the HuggingFaceπŸ€— community. Therefore we will use our 😊 heavily in this course.

On the next pages you can see what the content of each course week will be starting with what will happen during each of our course sessions. Then again the SmilingFace😊 will lead you to what else to do in the week. I have divided the course into three levels of course work:

😊

The part after one 😊 is mandatory for each course participant for a for successful participation

😊😊

The part after two 😊😊 is voluntary but recommended

😊😊😊

The part after three 😊😊😊 is completely voluntarily for the ones who really want to know

Remember the course instructor(me) is also fallible so please question me if you see something that does not kind of seem right for you. Also always ask questions especially if you don't fully understand something. This is really why we give this course so that you understand everything😊

Own contributions or suggestions for improving the course as well as feedback are always welcome😊

Let's dive right in!

Intermediate Machine Learning (Legacy SS2023)

Hybrid Format - Every Wednesday 18h00

Prequisites

There are certain requirements which form the basis for a successful course participation. If you do not have the mandatory requirements listed below, cosidering enrolling into a more basic course of our offerings. Alternatively bring yourself up to speed. Under additional ressources on the left sidebar you find the necessary ressources. Since the course has a really high pace it will be absolutely necessary to straighten these basic requirements before the course!

hashtag
Mandatory

Python

Here is a refresher notebook:

Math

Linear Algebra, Probability Theory (at least the basics)

Machine Learning

Basics:

  • What is a neural network

  • What is a forward/backprogragation

  • What is a loss

hashtag
Totally optional

You can set up your PC for local development. A guiding notebook is here:

Here different IDEs are presented and compared:

What is an activation function

Week 7 - Transformers Part 1

Understanding the Transformer

hashtag
Course session

Explanatory Session Part 1

Self-attention and multihead attention

Hugging Face Introduction

Library and Walk-through of HuggingFace101

Explanatory Session Part 2

Transformer Encoder and Positional Encoding

hashtag
To-do

😊

Go through this excellent site explaining Transformers:

Do Chapter 1 and Chapter 2 of the HuggingFace NLP course

Go through the TransformerHW1

😊😊

😊😊😊

Look closer at the Pytorch module nn.Transformer () and go through a on how to use it for next token prediction.

Week 2 - Recap ML Basics, Intro to PyTorch

hashtag
Course session

Quiz

ML Basics recap

Solutions exercises

Week 4 - Convolutional Neural Networks

CNNs

hashtag
Course session

Solutions exercise CNN

Presentation from the participants of the CNN assignment from Coursera

Presentation from the participants of the tasks from PyTorch101

Walk-through

PyTorch 202 (Lab 02)

hashtag
To-do

😊

Go for your own through the Colab Notebook above (Pytorch202) and try to understand and repeat the steps for your own.

Do Week 2 of the Coursera Course

The notebook from the Coursera Course of Week 2 can be accessed here:

The redundancy between our notebooks and the Coursera notebooks is desired to reintroduce the concepts in a different way and hence enrich your learning experience!

😊😊

Try to improve the accuracy in the PyTorch 202 notebook by tweaking the amount of layers and number of neurons

😊😊😊

Familiarize yourself with basic PyTorch Tutorials:

  • https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.htmlarrow-up-right (Second part)

  • https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial3/Activation_Functions.htmlarrow-up-right

  • https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial4/Optimization_and_Initialization.htmlarrow-up-right

Kaggle
  • Homework presentation of Logistic Regression for Paddy Disease Classification

Walk-through

Basic CNN in PyTorch:

PyTorch 404

Basic CNN in PyTorchLightning:

hashtag
To-do

😊

Go for your own through the Kaggle Notebook and PyTorch404 above and try to understand and repeat the steps for your own.

Do Week 4 of the Coursera Course

😊😊

Add the the test functionality and create a submission.csv and upload it to the leaderboard

😊😊😊

documentationarrow-up-right
tutorialarrow-up-right

Week 3 - Intro Kaggle competition - EDA and baseline models with PyTorch

Learning and testing - a.k.a. don't do Bullshit Machine Learning

hashtag
Course session

Kaggle

  • Introduction

  • Titanic

  • Paddy

  • Exploratory Data Analysis(EDA) for Paddy Disease Classification

Solutions exercise MLP

Presentation from the participants of the MLP from Coursera

Walk-through

PyTorchLightning

PyTorch 303 (Lab 03)

hashtag
To-do

😊

Go for your own through the Colab Notebook above (PyTorch303) and try to understand and repeat the steps for your own.

Do Week 3 of the Coursera Course

Please register at kaggle.com and join the competition. Go through the Exploratory Data Analysis Notebook session and then train a Logistic regression as baseline model!

The main objective of this Kaggle competition is to develop a machine or deep learning-based model to classify the given paddy leaf images accurately. A training dataset of 10,407 (75%) labeled images across ten classes (nine disease categories and normal leaf) is provided. Moreover, the competition host also provides additional metadata for each image, such as the paddy variety and age. Your task is to classify each paddy image in the given test dataset of 3,469 (25%) images into one of the nine disease categories or a normal leaf.

So that is where we will be heading in the next session trying different tools and techniques.

EDA Notebook

Logistic regression (try first on your own but if your stuck look at the notebook below):

😊😊

Build an MLP in PyTorchLightning for Paddy Challenge on Kaggle

😊😊😊

Do your own EDA on the Paddy Challenge and/or look at other EDA notebooks from competitors. Make a final presentable EDA notebook

Transfer the CNN from the Coursera assignment to our Kaggle competition

Familiarize yourself with this PyTorch Tutorials:

Week 6 - CNN and RNN Applications

Hands-on

hashtag
Course session

Kaggle Finetuning

Presentation of experiments with the goal of improving the classification accuracy

Week 5 - Recurrent Neural Networks

RNNs

hashtag
Course session

Faster Coding with ChatGPT, Stackoverflow and clever search

Solutions exercise RNN

Presentation from the participants of the RNN assignment from Coursera

Transfer Learning

Theory and Applications

hashtag
To-do

😊

Watch the second half of Week 5 of the Coursera Course

Watch the following Seminar about Transformers:

😊😊

Go on using ideas discussed in this session and go on improving the accuracy on the Paddy Dataset

Deep dive

  • What are Embeddings?

  • Reinforcements of and insights into RNNs beyond Coursera

Walk-through

PyTorch 505

Transfer Learning CNN in PyTorchLightning:

hashtag
To-do

😊

Watch first half of Week 5 of the Coursera Course

!!!

Using all techniques learned build the best model you can to achieve an accury at least above 70%. You can use Transfer Learning, Augmentation and other tricks. You can also take inspiration from fellow notebooks on Kaggle. Good ideas will be rewarded by special achievement badges for the course. Have fun and push the accuracy!😊

!!!

Week 10-12 - Projects sessions

Kaggle Competition

Google Colabcolab.research.google.comchevron-right
Logo

Week 1 - Course Introduction

hashtag
Course session

Welcome and Introduction round

Introduction of the course, opencampus, the course instructor and the course participants

Tool Set-Up

  • Coursera

  • Colab

  • Editor (VSCode)

  • Virtual Environments

  • Git/Github

Walk-through

PyTorch 101 (Lab 01)

A visual overview of the workflow in the Colab notebook you can get in the PyTorch diagram below:

hashtag
To-do

😊

  1. Watch the following introduction video to the PyTorch framework

  1. Watch Week 1 of the Coursera Course

  1. Go for your own through the Colab Notebook above (Pytorch101) and try to understand and repeat the steps for your own. Thereby you should also solve Task 1-3 in the notebook. You can create therefore a copy of the notebook in your Drive or download the notebook to work locally on it. Ensure that you sufficient computing resources available (i.e gpu) if you choose to work locally.

😊😊

Try to improve the accuracy in the PyTorch 101 notebook by tweaking the amount of layers and number of neurons

😊😊😊

Familiarize yourself with basic PyTorch Tutorials:

  • (First part)

file-pdf
97KB
pytorch diagram.pdf
PDF
arrow-up-right-from-squareOpen
https://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.htmlarrow-up-right
https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial2/Introduction_to_PyTorch.htmlarrow-up-right
Google Colabcolab.research.google.comchevron-right
Google Colabcolab.research.google.comchevron-right
Logo
Logo
Introduction - Hugging Face LLM Coursehuggingfacechevron-right
Logo
Google Colabcolab.research.google.comchevron-right
Logo
The Illustrated Transformerjalammar.github.iochevron-right
Introduction to Machine LearningCourserachevron-right
Introduction to Machine LearningCourserachevron-right
Introduction to Machine LearningCourserachevron-right
Introduction to Machine LearningCourserachevron-right
Introduction to Machine LearningCourserachevron-right
Logo
Tutorial 6: Transformers and Multi-Head Attention β€” UvA DL Notebooks v1.2 documentationuvadlc-notebooks.readthedocs.iochevron-right
Google Colabcolab.research.google.comchevron-right
Logo
Logo
Logo
Logo
Logo
Logo
Google Colabcolab.research.google.comchevron-right
The first notebook
Logo
Opencampus Basic CNN in PyTorchKagglechevron-right
Logo
Google Colabcolab.research.google.comchevron-right
The assignment notebook
Logo
Google Colabcolab.research.google.comchevron-right
Logo
Google Colabcolab.research.google.comchevron-right
Opencampus Paddy EDAKagglechevron-right
Opencampus Paddy EDAKagglechevron-right
Google Colabcolab.research.google.comchevron-right
Tutorial 5: Inception, ResNet and DenseNet β€” UvA DL Notebooks v1.2 documentationuvadlc-notebooks.readthedocs.iochevron-right
Tutorial 5: Inception, ResNet and DenseNet β€” UvA DL Notebooks v1.2 documentationuvadlc-notebooks.readthedocs.iochevron-right
Logo
Logo
Paddy Doctor: Paddy Disease Classificationwww.kaggle.comchevron-right
Paddy Doctor: Paddy Disease Classificationwww.kaggle.comchevron-right
Logo
Logo
Logo
Logo
Logo
Logo
Google Colabcolab.research.google.comchevron-right
Logo
Stanford CS25: V2 I Introduction to Transformers w/ Andrej KarpathyYouTubechevron-right

Week 9 - Vision Transformers

Apply Transformer to Vision

hashtag
Course session

Explanatory Session

Vision Transformer from Scratch

Walk-through

Finetuning Vision Transformer on Kaggle Paddy Dataset

hashtag
To-do

😊

Look at current Kaggle competitions and make proposals

😊😊

Logo

Week 13 - Project Presentations

Final Session

Week 8 - Transformers Part 2

Hugging Face

hashtag
Course session

Finetuning Trasformers for NLP with Hugging Face

hashtag
To-do

😊

Watch this video

😊😊

Do Week 3 of the Hugging Face NLP course

Stanford CS224N NLP with Deep Learning | 2023 | PyTorch Tutorial, Drew KaulYouTubechevron-right
Google Colabcolab.research.google.comchevron-right
Opencampus Paddy PyTorch Logistic RegressionKagglechevron-right
Logo
Logo
Logo

Week 14+

Go on with Kaggle Competition

Tutorial 15: Vision Transformers β€” UvA DL Notebooks v1.2 documentationuvadlc-notebooks.readthedocs.iochevron-right
Logo
Google Colabcolab.research.google.comchevron-right
Logo
Introduction - Hugging Face LLM Coursehuggingfacechevron-right
Logo
Google Colabcolab.research.google.comchevron-right
Logo
Google Colabcolab.research.google.comchevron-right
Logo