Deep learning course

This repo supplements Deep Learning course taught at YSDA and Skoltech @spring'19. For previous iteration visit the fall18 branch.

Lecture and seminar materials for each week are in ./week* folders. Homeworks are in ./homework* folders.

General info

• Create cloud jupyter session from this repo -
• Telegram chat room (russian).
• YSDA deadlines & admin stuff can be found at the YSDA course wiki (ysda students only).
• Any technical issues, ideas, bugs in course materials, contribution ideas - add an issue

Syllabus

• week01 Intro to deep learning

  • Lecture: Deep learning -- introduction, backpropagation algorithm
  • Seminar: Neural networks in numpy
  • Homework 1 is out!
  • Please begin worrying about installing pytorch. You will need it next week!

• week02 Adaptive optimization methods

  • Lecture: Empirical risk minimization, standard loss functions, linear classification, stochastic optimizers, adaptive SGD
  • Seminar: PyTorch basics

• week03 Convolutional networks I

  • Lecture: Convolutional networks (ConvNets), computer vision
  • Seminar: Convolutional networks in pytorch
  • Homework 2 is out!

• week04 Convolutional networks II

  • Lecture: ConvNet architectures, representations inside CNNs; visualizing networks/inceptionism, transfer learning
  • Seminar: Fine-tuning a pre-trained network

• week05 Advanced Computer vision

  • Lecture: "Deep" computer vision beyond classification; Verification tasks, object detection architectures, semantic segmentation
  • Seminar: Semantic segmentation
  • Homework 3 is out!

• week06 Deep generative models I

  • Lecture: Deep image generation; generative ConvNets, perceptual loss functions.
  • Seminar: Art Style Transfer by Dmitry Ulyanov

• week07 Deep generative models II

  • Lecture: Generative Adversarial Networks
  • Seminar: Generative Adversarial Networks

• week08 Unsupervised deep learning

  • Lecture: Autoencoders, variational autoencoders, image analogies
  • Seminar: Variational autoencoders

• week09 Deep learning for natural language processing

  • Lecture: Word embeddings, word2vec and other variants, convolutional networks for natural language
  • Seminar: Word embeddings. Text convolutions for salary prediction.
  • Homework 4 is out!

• week10 Recurrent neural networks

  • Lecture: Modelling sequences. Simple RNN. Why BPTT isn't worth 4 letters. GRU/LSTM.
  • Seminar: Generating human names and deep learning papers with RNNs

• week11 Recurrent neural networks II

  • Lecture: Sequence2sequence, architectures with attention and long-term memory.
  • Seminar: Image Captioning

• week12: Deep Reinforcement Learning

  • Lecture: Reinforcement Learning, MDPs, policy gradient methods
  • Seminar: REINFORCE on simple robot control, optional: advantage actor-critic on atari

Contributors & course staff

Course materials and teaching performed by

• Victor Lempitsky - all main track lectures (1-11)
• Victor Yurchenko - intro notebooks, admin stuff
• Vadim Lebedev - notebooks, admin stuff
• Dmitry Ulyanov - notebooks on generative models & autoencoders
• Fedor Ratnikov - pytorch & nlp notebooks, one bonus lecture
• Oleg Vasilev - notebooks, technical issue resolution
• Arseniy Ashukha - image captioning materials
• Mikhail Khalman - variational autoencoder materials