Subjects

Your browser does not support JavaScript, or its support is disabled. Some features may not be available.

Deep Learning - NPFL138

Title:	Hluboké učení
Guaranteed by:	Institute of Formal and Applied Linguistics (32-UFAL)
Faculty:	Faculty of Mathematics and Physics
Actual:	from 2023
Semester:	summer
E-Credits:	8
Hours per week, examination:	summer s.:3/4, C+Ex [HT]
Capacity:	unlimited
Min. number of students:	unlimited
4EU+:	no
Virtual mobility / capacity:	no
State of the course:	taught
Language:	Czech, English
Teaching methods:	full-time
Teaching methods:	full-time
Additional information:	http://ufal.mff.cuni.cz/courses/npfl138

Guarantor:	RNDr. Milan Straka, Ph.D.
Incompatibility :	NPFL114
Interchangeability :	NPFL114
Is incompatible with:	NPFL114
Is interchangeable with:	NPFL114

Opinion survey results Examination dates SS schedule Noticeboard

Annotation -

Last update: RNDr. Jiří Mírovský, Ph.D. (16.03.2024)

The objective of this course is to provide a comprehensive introduction to deep neural networks, which have consistently demonstrated superior performance across diverse domains. The course focuses both on theory spanning from the basics to the latest advances, as well as on practical implementations in Python and PyTorch (students implement and train deep neural networks performing a wide range of image processing, text processing, and image generation tasks). The course is part of the inter-university programme prg.ai Minor (https://prg.ai/minor).

Aim of the course -

Last update: RNDr. Jiří Mírovský, Ph.D. (11.05.2023)

The goal of the course is to introduce deep neural networks, from the basics to the latest advances. The course will focus both on theory as well as on practical aspects.

Course completion requirements -

Last update: RNDr. Jiří Mírovský, Ph.D. (11.05.2023)

Students pass the practicals by submitting sufficient number of assignments. The assignments are announced regularly the whole semester and are due in several weeks. Considering the rules for completing the practicals, it is not possible to retry passing it. Passing the practicals is not a requirement for going to the exam.

Literature -

Last update: RNDr. Jiří Mírovský, Ph.D. (11.05.2023)

Yoshua Bengio, Ian Goodfellow, Aaron Courville: Deep learning, MIT Press, In preparation.

Jürgen Schmidhuber: Deep learning in neural networks: An overview, Neural networks 61 (2015): 85-117.

Sepp Hochreiter, and Jürgen Schmidhuber: Long short-term memory, Neural computation 9.8 (1997): 1735-1780.

Requirements to the exam -

Last update: RNDr. Jiří Mírovský, Ph.D. (11.05.2023)

The exam is written and consists of questions randomly chosen from a publicly known list. The requirements of the exam correspond to the course syllabus, in the level of detail which was presented on the lectures.

Syllabus -

Last update: RNDr. Milan Straka, Ph.D. (07.02.2024)

Feedforward deep neural networks

Basic architectures and activation functions

Optimization algorithms for training deep models

Regularization of deep models

Classic regularization using parameter norm penalty

Dropout

Label smoothing

Batch normalization

Multi-task learning

Convolutional neural networks

Convolutional and pooling layers

Architectures suitable for very deep convolutional networks

State-of-the-art models for image recognition, object localization and image segmentation

Pre-training and finetuning of deep neural networks

Recurrent neural networks

Basic recurrent network, specifics of training

Long short-term memory

Gated recurrent units

Bidirectional and deep recurrent networks

Encoder-decoder sequence-to-sequence architectures

Practical methodology

Choosing suitable architecture

Hyperparameter selection

Natural language processing

Distributed word representations

Character-level word embeddings

Transformer architecture

State-of-the-art POS tagging, named entity recognition, machine translation

Deep generative models

Variational autoencoders

Generative adversarial networks

Diffusion models

Speech generation

Structured prediction

CTC loss and its application in state-of-the-art speech recognition

sequence-to-sequence architecture

Introduction to deep reinforcement learning

Entry requirements -

Last update: RNDr. Milan Straka, Ph.D. (08.11.2023)

Basic programming skills in Python are required. No previous knowledge of artificial neural networks is needed, but basic understanding of machine learning is advantageous.