David McAllester, April 2017

Lecture slides for 21 topics in deep learning with pointers into Deep Learning by Goodfellow, Bengio and Courville, as well as pointers to other relevant material.

1. Multi-Layer Perceptrons (MLPs) and Stochastic Gradient Descent (SGD)
2. Feed-Forward Computation Graphs, Backpropagation, and the Educational Framework (EDF)
3. Minibatching in EDF
4. Variants of SGD
5. An SGD Progress Theorem
6. Architecture and Universality
7. Convolutional Neural Networks (CNNs)
8. Some Linear Systems and Wavelet Theory
9. Second Order Optimization Methods
10. Vanishing Gradients, Xavier Initialization, Batch Normalization and Highway Architectures (Resnets, LSTMs and GRUs)
11. Regularization
12. Some Generalization Theory
13. Interpreting Deep Networks
14. Sequence to Sequence Models and Attention
15. Deep Reinforcement Learning
16. AlphaGo
17. Deep Graphical Models
18. Unsupervised and Predictive Learning
19. Information Theory and Distribution Modeling
20. Variational Autoencoders
21. A Rate-Distortion Case Study
22. Generative Adversarial Networks (GANs)