Neural Networks - NAIX002
Title: Neuronové sítě
Guaranteed by: Student Affairs Department (32-STUD)
Faculty: Faculty of Mathematics and Physics
Actual: from 2022
Semester: winter
E-Credits: 9
Hours per week, examination: winter s.:4/2, C+Ex [HT]
Capacity: unlimited
Min. number of students: unlimited
4EU+: no
Virtual mobility / capacity: no
State of the course: taught
Language: English
Teaching methods: full-time
Is provided by: NAIL002
Guarantor: doc. RNDr. Iveta Mrázová, CSc.
RNDr. František Mráz, CSc.
Class: Informatika Mgr. - Teoretická informatika
Classification: Informatics > Theoretical Computer Science
Pre-requisite : {NXXX038, NXXX039, NXXX040, NXXX067, NXXX069}
Incompatibility : NAIL002
Interchangeability : NAIL002
Is incompatible with: NAIL002
Is interchangeable with: NAIL002
Opinion survey results   Examination dates   WS schedule   Noticeboard   
Annotation -
The theory of neural networks is motivated by the results achieved in the area of the central neural system research. These inventions often represent the origin for the derived mathematical models which have (despite of significant simplifications of real neuro-physiological processes) some features of the natural intelligence. These models can be used in the design of non-traditional computational means applied in the solutions of many practical problems.
Last update: Zavoral Filip, RNDr., Ph.D. (03.04.2001)
Aim of the course -

To teach theory, algorithms and methods used in different models of artificial neural networks.

Last update: Mráz František, RNDr., CSc. (30.09.2020)
Course completion requirements -

A) The lab

Step by step, in an accompanying Moodle course, there will be published assignments and quizzes.

Assignments:

Each assignment has a deadline till which the assignment should be submitted for grading. A draft solution of an assignment can be edited at any time, but the time of submission is the time you click the button "Submit assignment". After clicking this button, you cannot edit your submission anymore, but you can ask (per e-mail) your teacher to return the assignment to the draft state. The teacher will grade each submitted assignment with 0–10 points. During the semester, you will solve three assignments.

A typical solution for an assignment will consist of a text – a description of the solution – and a Python code for solving the task. Both description and code can be submitted as a Jupyter notebook. Alternatively, you can submit your texts as a PDF file and the source codes as plain ASCII files.

Warning: If N≥2 participants of the course will submit solutions that are very similar or identical, all these solutions will be considered as a single solution. The solution will be graded by B points according to its quality and all students who submitted it will obtain only the integer part of the value B/N points.

Quizzes:

Beside the assignments, you will solve several on-line quizzes. You can obtain at most 10 points for the first so-called entrance test. During the semester, you can obtain 20 points from four short tests. Each quiz will have set up also a deadline. In contrast to assignments, it will be not possible to solve any quiz after its deadline.

For obtaining credits for the lab it is necessary:

  1. To solve all the assignments and to obtain at least 1 point for each solution. WARNING: late submission of a solution will be penalized by 1 point decrease for each started week of the delay after the deadline

  2. To prepare and to present a term project in a lab in the last week of this term or on a date (during the following exam period) that will be set up on a lab within the last week of this semester. Subjects for the project will be discussed in a lab in the middle of the term. Each project will be graded up to 15 points according to its quality.

The quizzes are not among the necessary conditions for obtaining credits for the lab. During labs, it is possible to obtain additional points

  • for demonstrating a solution of a problem assigned during a lab — 1 point,

  • for demonstrating a solution submitted as a solution for an assignment in Moodle (after its deadline) — the integer part of the half of the number of point awarded for the solution (after evaluating by the teacher)

Except for the additional points, obtaining up to 75 points is possible. All points obtained during labs will account for up to 35% of the final exam score. However, when a student gets more than 85 points within labs (counting also additional points), these points will still account for only 35% of the final score of the exam.

Continuous work throughout the whole term is required to obtain the credit. Therefore there will be no additional possibilities to acquire them later.

B) The lecture

The lecture will be given two times per week according to the schedule. As already mentioned above, points acquired within the lab will account for up to 35% of the final score for the exam. Further, there will be two online tests which will be written during labs. The dates for the tests will be set up within the first week of the semester and published in the accompanying Moodle course.

Each test will be graded with 0-10% toward the final score. The exam at the end of this term will add up to the remaining 45% to the final score. The following table gives the final grade according to the achieved score:

grade 1 grade 2 grade 3 failure
100%–86% 85%–71% 71%–56% less than 56%

Last update: Mráz František, RNDr., CSc. (05.10.2023)
Literature -

  1. Abu-Mostafa Y. S., Magdon-Ismail M., Lin H.-T.: Learning From Data: A Short Course, AMLbook.com, 2012

  2. Goldberg D. E.: Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading, Mass. 1989

  3. Haykin S.: Neural Networks and Learning Machines, 3rd Edition, Pearson, 2009

  4. Kohonen T.: Self-Organizing Maps, Springer-Verlag, 1995

  5. Rojas R.: Neural Networks: A Systematic Introduction, Springer-Verlag, Berlin, 1996

  6. Šíma J. and Neruda R.: Teoretické otázky neuronových sítí, Matfyz Press, Praha, 1997

Last update: Mrázová Iveta, doc. RNDr., CSc. (27.05.2020)
Requirements to the exam -

The exam consists of a written and oral part. The written part precedes the oral part. Failing the written part impacts failing the whole exam, i.e. the exam will be classified by the grade 4 (failed) and the exam will not continue with the oral part. When failing the oral part the next (reparative) attempt will consist again of both the written and oral parts. The final grade of the exam is set based on points awarded for the written and oral parts of the exam as well as on the points obtained for student’s work throughout the semester - see Course completion requirements.

The written part of the exam consists of three questions related to the syllabus of the lecture and/or material covered during the lab classes.

The requirements for the exam correspond to the syllabus of the lecture within the extent presented in the classes. In order to take part in the exam, it is necessary to obtain Final course credit.

Last update: Mrázová Iveta, doc. RNDr., CSc. (27.05.2020)
Syllabus -

1. Introduction to the area of artificial neural networks

  • Biological neuron and neural networks, transmission of signals in axons and synapses, information processing in neurons, main parts of the brain.

  • History and fundamental principles of artificial neural networks.

  • Adaptation and learning, a formal description of patterns.

  • Selection and ordering of pattern features, selection and ordering of training patterns.

2. Early models of artificial neural networks

  • The model of a formal neuron, weights, potential, transfer function.

  • Main types of artificial neural metworks.

  • Connectionism, training and recall, supervised learning and self-organization, knowledge extraction, generalization and robustness.

  • Perceptron and linear separability, separating hyperplane. Perceptron training algorithm and its convergence, the pocket algorithm.

3. Feed-forward neural networks and error back-propagation

  • The back-propagation training algorithm and the derivation of weight adjustment rules. Training, test and validation sets, various training strategies.

  • Internal knowledge representation, generalization, over-fitting and over-sizing, Vapnik-Chervonenkis dimension.

  • Kolmogorov´s theorem, function approximation, complexity of the learning problem.

  • Main areas and principles for applications of feed-forward neural networks.

4. Associative networks

  • Recurrent neural networks, Hebbian learning, memory capacity, attractors, energy function and convergence to stable states.

  • Associative memories, bidirectional associative memories (BAM), the Hopfield model, continuous Hopfield model, simulated annealing, the Boltzmann machine.

  • Hopfield networks in the search for suboptimal solutions of NP-complete problems.

5. Self-organization and hybrid models

  • Unsupervised reinforcement learning - Oja´s algorithm for PCA.

  • Kohonen self-organizing feature maps and algorithms for their training, lateral inhibition, topological neighborhood.

  • Counter-propagation neural networks, RBF-networks, Adaptive Resonance Theory (ART).

  • Cascade correlation and modular neural networks - mixtures of local experts.

6. Genetic algorithms

  • Coding of the optimization problem, population of strings, fundamental genetic operators - selection, cross-over, mutation.

  • Fitness function. Convergence analysis - schemata theorem.

  • Applications of genetic algorithms in the field of artificial neural networks.

Last update: Mráz František, RNDr., CSc. (30.09.2020)