SubjectsSubjects(version: 953)
Course, academic year 2023/2024
   Login via CAS
Deep Reinforcement Learning - NPFL139
Title: Hluboké zpětnovazební učení
Guaranteed by: Institute of Formal and Applied Linguistics (32-UFAL)
Faculty: Faculty of Mathematics and Physics
Actual: from 2023
Semester: summer
E-Credits: 8
Hours per week, examination: summer s.:3/4, C+Ex [HT]
Capacity: unlimited
Min. number of students: unlimited
4EU+: no
Virtual mobility / capacity: no
State of the course: taught
Language: Czech, English
Teaching methods: full-time
Teaching methods: full-time
Additional information: http://ufal.mff.cuni.cz/courses/npfl139
Guarantor: RNDr. Milan Straka, Ph.D.
Incompatibility : NPFL122
Interchangeability : NPFL122
Is incompatible with: NPFL122
Is interchangeable with: NPFL122
Annotation -
In recent years, reinforcement learning has been combined with deep neural networks, giving rise to game agents with super-human performance (for example for Go or chess, capable of being trained solely by self-play), datacenter cooling algorithms more efficient than human operators, or faster code for sorting or matrix multiplication. The goal of the course is to introduce reinforcement learning employing deep neural networks, focusing both on the theory and on practical implementations. The course is part of the inter-university programme prg.ai Minor (https://prg.ai/minor).
Last update: Mírovský Jiří, RNDr., Ph.D. (16.03.2024)
Aim of the course -

The goal of the course is to introduce reinforcement learning combined with deep neural networks. The course will focus both on theory as well as on practical aspects.

Last update: Mírovský Jiří, RNDr., Ph.D. (11.05.2023)
Course completion requirements -

Students pass the practicals by submitting sufficient number of assignments. The assignments are announced regularly the whole semester and are due in several weeks. Considering the rules for completing the practicals, it is not possible to retry passing it. Passing the practicals is not a requirement for going to the exam.

Last update: Mírovský Jiří, RNDr., Ph.D. (11.05.2023)
Literature -
  • Richard S. Sutton and Andrew G. Barto: Reinforcement Learning: An Introduction, Second edition, 2018.
  • David Silver et al.: Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm https://arxiv.org/abs/1712.01815
  • Julian Schrittwieser et al.: Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model https://arxiv.org/abs/1911.08265
Last update: Mírovský Jiří, RNDr., Ph.D. (11.05.2023)
Requirements to the exam -

The exam is written and consists of questions randomly chosen from a publicly known list. The requirements of the exam correspond to the course syllabus, in the level of detail which was presented on the lectures.

Last update: Mírovský Jiří, RNDr., Ph.D. (11.05.2023)
Syllabus -

Reinforcement learning framework

Tabular methods

  • Dynamic programming
  • Monte Carlo methods
  • Temporal-difference methods
  • N-step bootstrapping

Functional Approximation

Deep Q networks

Policy gradient methods

  • REINFORCE
  • REINFORCE with baseline
  • Actor-critic
  • Trust Region Policy Optimization
  • Proximal Policy Optimization

Continuous action domain

  • Deep Deterministic policy gradient
  • Twin Delayed Deep Deterministic policy gradient

Monte Carlo tree search

  • AlphaZero architecture

Model-based algorithms

  • MCTS with a learned model

Partially observable environments

Discrete variable optimization

Last update: Mírovský Jiří, RNDr., Ph.D. (11.05.2023)
Entry requirements -

Python programming skills and basic PyTorch/Tensorflow skills are required (the latter can be obtained on the Deep Learning NPFL138 course). No previous knowledge of reinforcement learning is necessary.

Last update: Straka Milan, RNDr., Ph.D. (09.11.2023)
 
Charles University | Information system of Charles University | http://www.cuni.cz/UKEN-329.html