DEEP LEARNING
DS-GA 1008 · SPRING 2020 · NYU CENTER FOR DATA SCIENCE
| INSTRUCTORS | Yann LeCun & Alfredo Canziani |
| LECTURES | Mondays 16:55 – 18:35, GCASL C95 |
| PRACTICA | Tuesdays 19:10 – 20:00, GCASL C95 |
| PIAZZA | Access code: DLSP20 |
| MATERIAL | Google Drive, Notebooks |
Description
This course concerns the latest techniques in deep learning and representation learning, focusing on supervised and unsupervised deep learning, embedding methods, metric learning, convolutional and recurrent nets, with applications to computer vision, natural language understanding, and speech recognition. The prerequisites include: DS-GA 1001 Intro to Data Science or a graduate-level machine learning course.
Lectures
Legend: 🖥 slides, 📓 Jupyter notebook, 🎥 YouTube video.
| Week | Format | Title | Resources |
|---|---|---|---|
| ① | Lecture | History and motivation | 🖥️ 🎥 |
| Evolution and DL | |||
| Practicum | Neural nets (NN) | 📓 📓 🎥 | |
| ② | Lecture | SGD and backprop | 🖥️ 🎥 |
| Backprop in practice | |||
| Practicum | NN training | 🖥 📓 📓 🎥 | |
| ③ | Lecture | Parameter transformation | 🖥️ 🎥 |
| CNN | |||
| Practicum | Natural signals' properties | 🖥 📓 🎥 | |
| ④ | Practicum | 1D convolutions | 📓 🎥 |
| ⑤ | Lecture | Optimisation I | 🖥️ 🎥 |
| Optimisation II | |||
| Practicum | CNN, autograd | 📓 📓 🎥 | |
| ⑥ | Lecture | CNN applications | 🖥️ 🖥️ 🎥 |
| RNNs and attention | |||
| Practicum | Training RNNs | 📓 📓 🎥 | |
| ⑦ | Lecture | Energy-Based Models | 🖥️ 🎥 |
| SSL, EBM | |||
| Practicum | Autoencoders | 🖥️ 📓 🎥 | |
| ⑧ | Lecture | Contrastive methods | 🖥️ 🎥 |
| Regularised latent | |||
| Practicum | Training VAEs | 🖥️ 📓 🎥 | |
| ⑨ | Lecture | Sparsity | 🖥️ 🎥 |
| Word model, GANs | |||
| Practicum | Training GANs | 🖥️ 📓 🎥 | |
| ⑩ | Lecture | CV SSL I | 🖥️ 🎥 |
| CV SSL II | |||
| Practicum | Predictive Control | 🖥️ 📓 🎥 | |
| ⑪ | Lecture | Activations | 🖥️ 🖥️ 🖥️ 🎥 |
| Losses | |||
| Practicum | PPUU | 🖥️ 📓 🎥 | |
| ⑫ | Lecture | DL for NLP I | 🖥️ 🎥 |
| DL for NLP II | |||
| Practicum | Attention & transformer | 🖥️ 📓 🎥 |
People
| Role | Photo | Contact | About |
|---|---|---|---|
| Instructor |  |
Yann LeCun yann@cs.nyu.edu |
Silver Professor in CS at NYU and Turing Award winner |
| Instructor |  |
Alfredo Canziani canziani@nyu.edu |
Asst. Prof. in CS at NYU |
| Assistant |  |
Mark Goldstein goldstein@nyu.edu |
PhD student in CS at NYU |
| Webmaster |  |
Zeming Lin zl2799@nyu.edu |
PhD student in CS at NYU |
| Grader |  |
Serkan Karakulak sk7685@nyu.edu |
|
| Grader | |
Raghav Jajodia rj1408@nyu.edu |
|
| Grader | |
Priyank Pathak pp1953@nyu.edu |
|
| Grader | |
Chiao-Hsun Wang chw371@nyu.edu |
|
| Grader | |
Pedro Vidal pmh314@nyu.edu |
|
| Grader | |
Bixing Yan by783@nyu.edu |