KMA-RL1

Based on the CS-234 course

Lecute 1 - Introduction to Reinforcement Learning

Additional Materials

• High level introduction: SB chapter 1
• Linear algebra review
• Probability review
• Python tutorial

Lecture discussion

• gym env
• saple factory

Lecture 2 - Tabular MDP planning

• SB chapter 3, 4.1-4.4

Lecture 3 - Tabular RL policy evaluation

• SB chapter 5.1, 5.5, 6.1-6.3
• David Silver's Lecture 4

Lecture 4 - Q-learning

• SB chapter 5.2, 5.4, 6.4-6.5, 6.7

Lecture (4, 5, 6) - RL with function approximation

• SB chapter 9.3, 9.6, 9.7

Lecture (7, 8) - Policy Search

• SB chapter 13
• David Silver lecture: Policy Gradient Methods
• John Schulman: Policy Gradients and Q-learning
• Pieter Abbeel: lecture1
• Pieter Abbeel: lecture2
• Pieter Abbeel: lecture3
• Evolution strategies as a scalable alternative to reinforcement learning

Practical part

Practice-related

• RL specialization
• detailed practical materials
• previous-year assignemnts & solutions
• gymnasium gym deep-dive repo
• gym setup article

Recordings

• practice-01
• practice-02
• practice-03
• practice-04
• practice-05

Homeworks

Homework 1 (1, 2, 3)

Submission link
First deadline: 6/03/22 (30 points max)
Second deadline: 13/03/22 (25 points max)
Final deadline: 24/04/22 (20 points max)

Homework 2 (4, 5, 6)

Submission link
First deadline:
Second deadline:
Final deadline: 24/04/22

Homework 3 (7, 8)

Submission link
First deadline:
Second deadline:
Final deadline: 24/04/22

List of papers

• 13/04/22 Playing Atari with Deep Reinforcement Learning

• SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards

• Augmented Q Imitation Learning (AQIL)

• Proximal Policy Optimization Algorithms

• Training language models to follow instructions with human feedback

• AdaFrame: Adaptive Frame Selection for Fast Video Recognition; Adaptive Focus for Efficient Video Recognition; AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognition

• Benchmarking Reinforcement Learning Algorithms on Real-World Robots

• Generative Adversarial Imitation Learning

• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor