Since Feb 2022 I’ve been writing our textbook on Deep Learning with an Energy perspective. It will come in two versions: an electronic one with a dark background for screens (freely available) and a physical one with white background for printers (for purchase).

I finished writing the first 3 chapters and corresponding Jupyter Notebooks:

  • Intro;
  • Spiral;
  • Ellipse.

Once the 4th chapter and notebook are done (end of Aug?), the draft will be submitted to the reviewers (Mikael Henaff and Yann LeCun). After merging their contributions (end of Sep?), a first draft of the book will be available to the public on this website.

Book format

The book is highly illustrated using $\LaTeX$’s packages TikZ and PGFPlots. The figures are numerically generated, with the computations done in Python using the PyTorch library. The output of such computations is stored as ASCII files and then read by $\LaTeX$ that visualises them. Moreover, most figures are also rendered on the Notebook using the Matplotlib library.

Why plotting with $\LaTeX$?

Because I can control every single aspect of what is drawn. If I define the hidden vector $\green{\vect{h}} \in \green{\mathcal{H}}$ in the book, I can have a pair of axes lebelled $\green{h_1}$ and $\green{h_2}$ and the Cartesian plane labelled $\green{\mathcal{H}}$ without going (too) crazy. All my maths macros, symbols, font, font size, and colour are just controlled by one single stylesheet called maths-preamble.tex.

Why colours

Because I think in colours. Hence, I write in colours. And if you’ve been my student, you already know that at the bottom left we’ll have a pink-bold-ex $\pink{\vect{x}}$ from which we may want to predict a blue-bold-why $\blue{\vect{y}}$ and there may be lurking an orange-bold-zed $\orange{\vect{z}}$.

Illustrations sneak peeks

To keep myself motivated and avoid going crazy too much, I post the most painful drawings on Twitter, where my followers keep me sane by sending a copious amount of love ❤️. You can find here a few of these tweets.

Load tweets (may take a few seconds)

Last update: 26 Jul 2022.

Oct 2022 update

For the entire month of Aug and half of Sep I got stuck on implementing a working sparse coding algo for a low-dimensional toy example. Nothing was working for a long while, although I managed to get the expected result (see tweets below). Then, I spent a couple of weeks on the new semester’s lectures, creating new content (slides below, video available soon) on back-propagation, which I’ve never taught at NYU, a topic that will make it to the book. Anyhow, now I’m back to writing! 🤓

Load tweets (may take a few seconds)

Last update: 26 Sep 2022.

May 2023 update

Oh boy, this 4th chapter took me a while (mostly because I’ve focussed also on other things, including the Spring 2023 edition of the course)… but it’s done now! In these last few months I’ve written about undercomplete autoencoders (AE), denoising AE, variational AE, contractive AE, and generative adversarial nets. Thanks to Gabriel Peyré, I’ve developed a method to separate stationary sinks and sources for a dynamics field (which I may write an article about), and it’s an integral part of the book explanations.

Moreover, I’ve been pushing a few videos from the Fall 2022 edition of the course, which give a preview on the chapters I’ve been writing, e.g. neural nets components, backpropagation (first time teaching it), energy-based classification, PyTorch training, K-means, and sparse coding (at least for now). Finally, over the Winter break, I’ve been teaching 12 years-olds about the maths and computer science behind generative AI, and I’m considering using p5.js as a tool to teach programming to beginners.

What’s next? I’m sending this first draft, with its 4 chapters (Intro, Spiral, Ellipse, Generative) and companion Jupyter Notebooks to Yann for a review. Meanwhile, I’ll be writing down the Backprop chapter, possibly and article, and pushing a few more videos on YouTube. Once the review is completed, a first draft will pop to this website for the public.

Load tweets (may take a few seconds)

Figures from chapter 4

Videos from DLFL22

Teaching Italian 7th graders

Last update: 16 May 2023.

Aug 2023 update

Of course, during the Summer it was unrealistic expecting anyone to review anything… Anyhow, I’ve just got back from O‘ahu (ICML23) and Maui (2 days before Lahaina burnt down) and finished the Backprop chapter, therefore the first draft will have 5 chapters in total as of right now. Below, you can see a few diagrams I’ve developed over these summer months.

The new semester starts in two weeks, so I’ll be a bit busy with that. I need to plan a possible chapter on joint embedding methods and start working on PART II of the book: ‘geometric stuff’.

About books, I’ve just received my copy of The Little Book of Deep Learning by François Fleuret. I have to say it is really well-made, and I really like it. It’s a bit on the terse side, but I haven’t decided if it’s a pro or a con.

Load tweets (may take a few seconds)

Jun 2025 update

Oh boy… it’s been two years since the last update… Let met tell you what’s happened since the last time I wrote something here.

Autumn 2023

We left off at draft v0.5.0, with 5 chapters completed (Backprop being the last one). During autumn 2023 I completed the 6th chapter (Signals, draft v0.6.0), and started working on recurrent nets.

Load tweets (may take a few seconds)

My coworker, Brian McFee publishes Digital Signals Theory, an introductory textbook for non-technical people.

Spring 2024

I’m promoted to full-time teaching faculty, with two courses a semester. More precisely, I’m co-teaching a (classical?) symbolic and statistical AI course (don’t ask) to 130 students with no teaching assistant. (In addition to my 80-student graduate Deep Learning course, for a gran total of 210 students.) Therefore, I start working 7 days a week, 12 hours a day.

We decide to split the duties across the semester: I’m in charge of the second ‘learning’ part.

Now the fun part. Students don’t come to class (it’s not necessary for solving the first part’s homework), my slides do not have text, students complain they cannot ‘read’ the slides by their own, my exam is about the knowledge covered in class, I get a tonne of negative reviews on Rate My Professor. A few months later, I become the target of several angry, hateful students. I almost lost my job.

Book? What book? Who has time to focus on anything else?

Yet, I publish some of my lectures as NYU-AISP24.

Summer 2024

I interview Yann LeCun and Léon Bottou. I put together 3 blog posts on SN, Yann and Léon’s 1988 Simulateur de Neurones learning framework. Furthermore, I write a blog about ‘visual requirements’ for my grad course.

Load tweets (may take a few seconds)

Autumn 2024

I’m still teaching two courses a semester, but one is an offering for alumni of my graduate Deep Learning course. Therefore, there’s minimal overhead and I can get back to writing.

I complete the History (7th) chapter (draft v0.7.0). I push a little more, and complete the RNN (8th) chapter (draft v0.8.0). Finally, I start the TikZ (9th) chapter, where I explain how I draw all my book’s figures.

I become a hate crime victim, target of a psychopath, who verbally threats me. I fear for my safety and file a police report. In retaliation, the psychopath proceeds with a defamation campaign, trying to destroy my public figure.

Load tweets (may take a few seconds)

Spring 2025

I’m given the opportunity to decide what my second course is. Therefore, I put together an undergraduate ‘Introduction to Deep Learning’ blackboard course. Mum gifts me a chalk holder. I have zero registered students one week prior to the beginning of the semester. The admin tells me they will likely have to cancel my course, and I’ll be assigned some other random stuff to teach.

We’re having fun in class. Students are easily amused by this silly prof.

For the second lecture, I spend roughly 4 hours tweaking my slides. I go to class and decided to give an introduction before turning on my laptop and the projector. One hour and a half later… the blackboard is a copy of the slides I planned to use 😅

Load tweets (may take a few seconds)

This is fun! 🤩 I get the hang of it and start crafting live coloured blackboard. Students are enthusiastic and hyped.

Load tweet

I go teach in Santiago of Chile for Khipu 2025, and I get Yann to cover for me.

Load tweet

I get back to FPGA Verilog programming, Spice CMOS simulation, and digital electronics.

Load tweets (may take a few seconds)

In class, I experiment a lot with the guided discovery pedagogical technique and having the students being the major actors, to a point that lecture 20 got completely derailed by a student, who kept steering the thread, prompted by his own curiosity. I was so ecstatic about the outcome (it was pure jazz), that I decided to publish the lecture to advertise the course to other students.

Load tweet

Finally, I create a new animation about training a neural network for classification, reviving code written 5 years ago.

Load tweet

For this course, I had the pleasure to have an unofficial assistant, Gabriele Pintus, who has been writing his Master’s thesis on JEPA models with me, here at NYU. Thanks to him, the homework were spectacularly well-made, and students extremely happy.

Book? No time.

Summer 2025

Yann agrees to review the book in July, finally allowing me to release the first book’s draft. I complete and release the 9th chapter, TikZ, bumping the draft to v0.9.0. The next update should happen around the end of July, where I should be able to share with you the first draft of the book. Now, I’m getting started with the 10th chapter, Control.

Last update: 9 Jun 2025.