arrow-left

All pages
gitbookPowered by GitBook
1 of 15

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Loading...

Week 4 - Deep Neural Networks

hashtag
This week you will...

  • Explore deep neural network.

  • First example of generalizing a neural network with L layers.

  • Discussion and choice about the projects.

hashtag
Learning Resources

hashtag
Projects

hashtag
Until next week you should...

  • Create a group for your project!

  • Finish the first week of the

  • Do the Programming Assignments: Initialization, Regularization and Gradient Checking

Improving Deep Neural Network Coursearrow-up-right

Requirements for a Certificate of Achievement or ECTS

The conditions to be met in order to receive a Certificate of Achievement and the ECTS are:

  • Attendance to at least 80% of the classes (it is allowed to miss maximum 2 times)

  • Delivery of the project with the needed documentation.

hashtag
Attendance:

Since the weekly session will be on Zoom, please use your full name when you join the Zoom Meeting. The full name should be the same that you used to register at edu.opencampus.sh, because we have an automatic check.

triangle-exclamation

We register automatically the attendance.

When you join the Zoom Session, please use the same name you have in the edu.opencampus.sh platform. You can change your name in the edu.opencampus.sh platform (click at the top-right on your profile photo) and in Zoom (click at the top-right of your video stream), so you should be able to use the same name during the weekly session.

If for any reason (no need to explain) you do not want to use the same name, but still need to be registered, please contact me at the beginning of the course.

hashtag
Projects:

Check the Projects section to learn more about the projects.

hashtag
Coursera:

Each weekly session is complemented with the videos and homework from the Coursera courses. Going through the video and doing the assignment allows you to learn and understand each session, so it is required for the course.

However, Coursera is indipendent from us and the completion of the Coursera assignment is NOT needed for the Opencampus Certificate. Completing all assignment will give you the Coursera Certificates (which is different)

Course Projectschevron-right

Deep Learning from Scratch

Opencampus Course about Deep Learning based on various Coursera Courses

The Deep Learning course will guide you through the mathematics background of machine learning approaches. We will start from a simple neural network and go through the different components of a network to understand and be able to create your own project.

hashtag
The Objective of the Course

The aim of this course is to develop a deeper understanding of how and why neural network work. The first part will be devoted to understanding and implementing the basic behind most of the neural network approaches, the forward- and back-propagation, loss function, optimization, training, hyper-parameters tuning and analysis.

To gain a better understanding, we will implement those part in python (mostly using the numpy library). These methods already exists in popular frameworks (like Tensorflow or Pytorch, to cite a few), but using them without knowledge may be confusing.

During the course, we will have weekly discussion to deeper our understanding of the subject and also you will work on your own project.

hashtag
Requirements and Motivation

In order to get the best out of this course, some previous knowledge is required. We expect the participant to have an understanding about the fundaments of mathematics (not being afraid of derivatives), linear algebra (mostly matrix multiplication) and python programming.

Based on the past semester, the estimated time is around 5 hours a week, ranging usually from 3 to 8 depending on the week's material. The project will start after 4 weeks of the course and will take some additional hours. However, assignment will decrease in the end of the course to leave you space for the project. Be sure to allocate enough time to manage to get through the whole course. If in doubt, ask us for advice.

circle-info

You do not have to be an expert, and sometimes enthusiasm and motivation may be enough. If you are unsure about some of the requirements, check out the Additional Resources or write us to discuss about it.

hashtag
The Project

Groups of students should be formed to work on a project. The project idea can come from the student, from a template or proposed from us. The project is needed in order to finish the course, and a final presentation will be given in the last week of the course.

circle-info

For the complete requirements about the project, check out the Requirement page.

circle-info

For some example of projects from last years, check out the Past Projects page.

hashtag
The Course Material

The course will be held weekly and will constitute of an online session of 1 hour and a half. The material and slides for each session are found in each week's page.

hashtag
ECTS

For further details about Certificates and ECTS please refer to the following page:

Learning Linear Algebrachevron-right
Learning Pythonchevron-right
How to Start, Complete, and Submit Your Projectchevron-right
Requirements for a Certificate of Achievement or ECTSchevron-right

Preparation

hashtag
Before the first class you should ...

  • Register yourself in the Opencampus Mattermost Chat

  • and for the , and enroll at least in the first course .

Register yourself in Courseraarrow-up-right
Deep Learning Specializationarrow-up-right
Neural Neworks and Deep Learningarrow-up-right

Week 5 - Practical Aspects of Deep Learning

hashtag
This week you will..

Get practical hints about initialization and regularization techniques to avoid overfitting and improving the training of a neural network.

hashtag
Learning Resources

hashtag
Slides

hashtag
Until next week you should...

  • Form the groups, decide the project and communicate it to the teacher.

  • Finish the second week of the

  • Do the Programming Assignment

hashtag

Improving Deep Neural Network Coursearrow-up-right

Week 1 - General Introduction

A general introduction about the course structure and the participants

hashtag
This week you will...

  • Receive an introduction about the course and the people in it. A short overview of the course, contents and how it will work.

  • Information about accounts, forum and contacts are provided.

hashtag
Learning Resources

hashtag
Until next week you should...

  • Register in the

  • Register on Coursera and start the course,

  • Finish the first two weeks of the

Do the Programming Assignment on Logistic Regression

  • Do the Programming Assignment on Python Numpy

  • Deep Learning Channel in the Mattermost Chatarrow-up-right
    Neural Networks and Deep Learningarrow-up-right
    coursearrow-up-right

    Week 2 - Introduction to Deep Learning and Neural Network Basics

    hashtag
    This week you will...

    • Check if everything worked with the tools we started using

    • Have the first session with a small quiz and round of discussion.

    • Discuss about python environment, dot product against element wise multiplication,

    • Do you first exercise session training a small neural network recognizing cats!

    hashtag
    Learning Resources

    hashtag
    Slides

    hashtag
    Useful (external) Resourcers

    hashtag
    Until next week you should...

    Week 10 - Bonus: most voted topic

    Week 3 - Shallow Neural Networks

    hashtag
    This week you will...

    check weights initialization in the training and notebook example of planar data classification changing the number of hidden unit in a shallow network - only 1 hidden layer.

    hashtag
    Learning Resources

    hashtag
    Slides

    hashtag
    Useful (external) Resources

    hashtag
    Until next week you should...

    • Finish the

    • Finish both assignments

    • Think about your project: prepare an idea and find other people willing to collaborate (there is time also next week, but please start)

    fourth week of the coursearrow-up-right

    Week 11 - Presentation of Final Projects, Part I

    This session is entirely dedicated to the presentation of the final project from the students. Schedule and timing will be decided and published during the course.

    Week 12 - Presentation of Final Projects, Part II

    Week 4 - Deep Learning from Scratch @ Opencampusdeeplearning.freelab.orgchevron-right
    GitHub - opencampus-sh/project_template_folder: A template folder that you can download and fill with the necessary information to upload the project from the Courses belonging to the Opencampus Machine Learning DegreeGitHubchevron-right
    GitHub - opencampus-sh/bakery-sales-project: The template adapted with data and presentation for the bakery sales projectGitHubchevron-right

    Week 9 - Neural Networks Architecture | Project Checkpoint

    hashtag
    This week you will..

    • Get an overview about the different architectures

    • Discuss the status of your project

    Week 8 - Machine Learning Strategy 1 & 2

    hashtag
    This week you will..

    • Guidelines for the projects presentations, suggestions on what to put on them, baseline and human performances.

    • Talk about how to structure the training, test and validation set, and more general on how to structure the whole project.

    • What about using transfer learning, end-to-end approaches, divide the problem into smaller subproblems, or using multi-tasking? Sometimes the problem can be seen from another perspective.

    • Since there were no programming assignment for this week, walkthrough an LSTM tutorial on time series.

    hashtag
    Learning Resources

    hashtag
    Slides

    Logo
    Logo
    Week 6 - Deep Learning from Scratch @ Opencampus - SoSe 2021deeplearning.freelab.orgchevron-right
    Week 2 - Deep Learning from Scratch @ Opencampus.shdeeplearning.freelab.orgchevron-right

    Week 7 - Hyperparameter Tuning

    hashtag
    This week you will..

    Quick overview of the projects, report from Hackathon, discussion about batch normalization and hyperparameters search, first assignment using Tensorflow to create a small neural network. Small discussion about Tensorflow ideas and modalities and difference between 1.0 and 2.0 versions.

    hashtag
    Learning Resources

    hashtag
    Slides

    Week 1 - Deep Learning @ Opencampus.shdeeplearning.freelab.orgchevron-right
    Károly Zsolnai-Fehér - Research ScientistKároly Zsolnai-Fehér - Research Scientistchevron-right
    Trending Papers - Hugging Facehuggingfacechevron-right
    Logo
    Lecturescs230.stanford.educhevron-right
    Logo
    The most insightful stories about Deep Learning - MediumMediumchevron-right
    Deep Learningwww.deeplearningbook.orgchevron-right
    Week 3 - Deep Learning from Scratch @ Opencampusdeeplearning.freelab.orgchevron-right
    Logo
    Logo
    Logo
    Logo
    https://unity3d.com/machine-learningunity3d.comchevron-right
    Unity toolkit for Machine Learning
    opencampus.sh machine learning projects | opencampus.sh machine learning projectsopencampus-sh.github.iochevron-right
    Projects from the last semesters
    Logo
    Kaggle: Your Machine Learning and Data Science CommunityKagglechevron-right
    Logo
    Week 9 - Deep Learning @ Opencampusdeeplearning.freelab.orgchevron-right

    Week 6 - Optimization Algorithms

    hashtag
    This week you will..

    See Mini-batches, Momentum, RMSProp and AdamOptimizer: an overview of optimization algorithm to train faster neural networks.

    hashtag
    Learning Resources

    hashtag
    Slides

    Week 7 - Deep Learning from Scratch @ Opencampus - SoSe 2021deeplearning.freelab.orgchevron-right
    Logo