Introduction to Deep Learning Research

CSCI-UA 480 075 Β· FALL 2025 Β· NYU COURANT INSTITUTE OF MATHEMATICAL SCIENCES

INSTRUCTOR Alfredo Canziani
LECTURES Tue/Thu 12:00 – 13:45
CODE 2025 repo
BLACKBOARDS Google Drive
READINGS Google Drive
SLIDES Google Drive

This second offering of my new course is meant to be an introduction to Deep Learning research for undergraduate (or advanced high school) students.

The aim of this course is to get the students fluent in reasoning, using:

  • maths (linear algebra, calculus, logic),
  • diagrams and schematics (abstract graphical language),
  • graphs (function plotting and asymptotic behaviour),
  • physics (reducing systems to their base parts to identify emerging collective behaviours), and
  • coding (empirical verification of proposed hypothesis).

To test the students’ knowledge, the course uses 6 quizzes throughout the semester, 4 homework assignments, 2 projects, and a final oral exam, where students are examined on final project significance and originality, project presentation and defence, course content knowledge, and communication effectiveness.

Selected final projects and code written in class can be found in the GitHub repo; slides, blackboards, and suggested readings can be found on Google Drive. All links are provided at the top of this web page.

Lectures

Legend: πŸ–₯ slides, πŸ“ notes, πŸ““ Jupyter notebook, πŸŽ₯ YouTube video.

Lesson 01 πŸŽ₯

Course intro + McCulloch & Pitts binary neuron
Using maths & coding as languages of research πŸ“πŸ’»

Suggested readings

Suggested videos

Lesson 01 blackboard

Lesson 02 πŸŽ₯

Programming a neural network
Behaviour by design using weights computed with maths πŸ“πŸ§ 

Suggested readings

Lesson 02 blackboard

Lesson 03 πŸŽ₯

Wiener's cybernetics, Hebbian plasticity, and Rosenblatt's perceptron
When physical machines start learning πŸ”

Suggested readings

Lesson 03 blackboard 1 Lesson 03 blackboard 2

Lesson 04 πŸŽ₯

Bias, perceptron properties, and multi-class classification
Bias shifts the boundary; more neurons slice the world πŸ“πŸ§ 

Lesson 04 blackboard 1 Lesson 04 blackboard 2

Lesson 05 πŸŽ₯

A softer perceptron, part I: probabilities
Replacing certainty πŸŒ— with a degree of belonging πŸ“Š

Suggested readings

Lesson 05 blackboard 1 Lesson 05 blackboard 2

Lesson 06 πŸŽ₯

A softer perceptron, part II: likelihood and loss
Cross-entropy turns belief into a training signal πŸ“πŸ“Š

Suggested readings

Lesson 06 blackboard 1 Lesson 06 blackboard 2

Lesson 07 πŸŽ₯

A softer perceptron, part III: gradient descent
One βˆ‡ vector, two answers: where to go 🧭 and how fast πŸƒπŸ’¨

Suggested readings

Lesson 07 blackboard 1 Lesson 07 blackboard 2

Lesson 08

A softer perceptron, part IV: hardening and multi-class

Lesson 09

A softer perceptron, part V: multi-class likelihood and loss

Lesson 10

A softer perceptron, part VI: soft-stuff and multi-class SGD

Lesson 11

Loss zoo and the least-squares solution

Lesson 12

Adaline, first NN winter, and adaptive filters for system identification

Lesson 13

Inverse modelling with adaptive filters, 1980s historical background

Lesson 14

Learning the feature vector with back-propagation

Lesson 15

N per-sample losses and the back-propagation algorithm

Lesson 16

Backprop example, on the blackboard and with a Python class

Lesson 17

Gradient accumulation

Lesson 18

Learning the feature vector, part V: spiral 'despiralisation'

Lesson 19

Nonlinear classification with neural nets

Lesson 20

Supervised learning with PyTorch

Lesson 21

Natural signals and convolutional neural networks

Lesson 22

ConvNets for 2D signals, history, and recurrent nets

Lesson 23

Project 1: digit captioning

Lesson 24

Statistical Natural Language Processing (NLP)

Lesson 25

Neural NLP

Lesson 26

Attention-based NLP