SubjectsSubjects(version: 806)
Course, academic year 2017/2018
   Login via CAS
Neural Networks - NAIL002
Czech title: Neuronové sítě
Guaranteed by: Department of Theoretical Computer Science and Mathematical Logic (32-KTIML)
Faculty: Faculty of Mathematics and Physics
Actual: from 2014
Semester: winter
E-Credits: 9
Hours per week, examination: winter s.:4/2 C+Ex [hours/week]
Capacity: unlimited
Min. number of students: unlimited
State of the course: taught
Language: Czech, English
Teaching methods: full-time
Guarantor: doc. RNDr. Iveta Mrázová, CSc.
RNDr. František Mráz, CSc.
Class: Informatika Mgr. - Teoretická informatika
Classification: Informatics > Theoretical Computer Science
Annotation -
Last update: RNDr. Filip Zavoral, Ph.D. (03.04.2001)

The theory of neural networks is motivated by the results achieved in the area of the central neural system research. These inventions often represent the origin for the derived mathematical models which have (despite of significant simplifications of real neuro-physiological processes) some features of the natural intelligence. These models can be used in the design of non-traditional computational means applied in the solutions of many practical problems.
Aim of the course - Czech
Last update: T_KTI (26.05.2008)

Naučit teorii, algoritmy a používané metody pro různé architektury neuronových sítí.

Terms of passing the course -
Last update: RNDr. František Mráz, CSc. (10.10.2017)

A) The seminary

Step by step, in an accompanying Moodle course there will be published assignments and quizzes.

Assignments:

Each assignment has a deadline till which the assignment should be submitted for grading. A draft solution of an assignment can be edited at any time, but the time of submission is the time you click the button "Submit assignment". After clicking this button you cannot edit your submission anymore, but you can ask (per e-mail) your teacher to return the assignment back into the draft state. Each submitted assignment will be graded by the teacher with 0-10 points. During the semester, you will solve 4 assignments.

A typical solution for an assignment will consist of a text - a description of the solution - and a code of a program/script used for solving the assignment. Submit your texts as a PDF-file or alternatively as an RTF-file. The source codes should be submitted as plain ASCII files.

Warning: If N≥2 participants of the course will submit solutions which are very similar or identical, all these solutions will be considered as a single solution. The solution will be graded by B points according to its quality and all students who submitted it will obtain only the integer part of the value B/N points.

Quizzes:

Beside the assignments, you will solve several on-line quizzesFor the first so-called entrance test you can obtain at most 10 points. During the term there will be assigned several short tests for at most 10 points altogether. Each quiz will have set up also a deadline. In contrast to assignments it will be not possible to solve any quiz after its deadline. On the other hand, you will be given 3 attempts for solving each quiz and you will score the best score of all your attempts on the quiz.

For obtaining credits for the seminary it is necessary:

  1. To solve all the assignments and to obtain at least 1 point for each solution. WARNING: a late submission of a solution will be penalized by 1 point for each started week of the delay after the deadline.
  2. To prepare and to present a term project in a seminary in the last week of this term or on a date (during the following exam period) which will be set-up on a seminary within the last week of this term. Subject for the project will be discussed in a seminary in the middle of this term. Each project will be graded up to 15 points according to its quality.

The quizzes are not among the necessary conditions for obtaining credits for the seminary. During seminaries it is possible to obtain additional points

  • for demonstrating a solution of a problem assigned during a seminary  - 1 point,
  • for demonstrating a solution submitted as a solutions for an assignment in Moodle (after its deadline) - the integer part of the half of the number of point awarded for the solution (after evaluating by the teacher)

Except for the additional points, it is possible to obtain up to 75 points. All points obtained during seminaries will be accounted for up to 35% of the final score of the exam. However, when a student will obtain more than 75 points within seminaries (counting also additional points), these points will still account for only 35% of the final score of the exam.

Continuous work throughout the whole term is required to obtain the credits, therefore there will be no additional possibilities to acquire them later.

B) The lecture

The lecture will be given two-times per week according to the schedule. As already mentioned above, points acquired within the seminary will account for up to 35% of the final score for the exam. Further, there will be two written tests which will be written during lectures on the following dates

  • November 15, 2017 and
  • December 20, 2017

Each of these tests will be graded with 0-10% towards the final score. The exam at the end of this term will add up to the remaining 45% to the final score. The following table gives the final grade according to achieved score:

grade 1 grade 2 grade 3 failure
100%–86% 85%–71% 71%–56% less than 56%

Literature - Czech
Last update: doc. RNDr. Iveta Mrázová, CSc. (30.04.2015)

Abu-Mostafa Y. S., Magdon-Ismail M., Lin H.-T.: Learning From Data: A Short Course, AMLbook.com, 2012

Goldberg D. E.: Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Reading, Mass. 1989

Haykin S.: Neural Networks and Learning Machines, 3rd Edition, Pearson, 2009

Kohonen T.: Self-Organizing Maps, Springer-Verlag, 1995

Rojas R.: Neural Networks: A Systematic Introduction, Springer-Verlag, Berlin, 1996

Šíma J. and Neruda R.: Teoretické otázky neuronových sítí, Matfyz Press, Praha, 1997

Requirements to the exam -
Last update: RNDr. František Mráz, CSc. (14.10.2017)

The exam consists of a written and oral part. The written part precedes the oral part. Failing the written part impacts failing the whole exam, i.e. the exam will be classified by the grade 4 (failed) and the exam will not continue by the oral part. When failing the oral part the next (reparative) attempt will consist again of both the written and oral parts. The final grade of the exam is set based on points awarded for the written and oral parts of the exam as well as on the points obtained for student’s work throughout the semester - see Terms of passing the course.

The written part of the exam consists of three questions related to the syllabus of the lecture and/or material covered during the lab classes.

The requirements for the exam correspond to the syllabus of the lecture within the extent presented in the classes. In order to take part in the exam, it is necessary to obtain Final course credit.

Syllabus -
Last update: doc. RNDr. Iveta Mrázová, CSc. (06.05.2015)

1. Introduction to the area of artificial neural networks

  • Biological neuron and neural networks, transmission of signals in axons and synapses, information processing in neurons, main parts of the brain.
  • History and fundamental principles of artificial neural networks.
  • Adaptation and learning, a formal description of patterns.
  • Selection and ordering of pattern features, selection and ordering of training patterns. Principal Component Analysis.

2. Early models of artificial neural networks

  • The model of a formal neuron, weights, potential, transfer function.
  • Main types of artificial neural metworks.
  • Connectionism, training and recall, supervised learning and self-organization, knowledge extraction, generalization and robustness.
  • Perceptron and linear separability, separating hyperplane. Perceptron training algorithm and its convergence, the pocket algorithm.

3. Feed-forward neural networks and error back-propagation

  • The back-propagation training algorithm and the derivation of weight adjustment rules. Training, test and validation sets, various training strategies.
  • Internal knowledge representation, generalization, over-fitting and over-sizing, Vapnik-Chervonenkis dimension.
  • Kolmogorov´s theorem, function approximation, complexity of the learning problem.
  • Main areas and principles for applications of feed-forward neural networks.

4. Associative networks

  • Recurrent neural networks, Hebbian learning, memory capacity, attractors, energy function and convergence to stable states.
  • Associative memories, bidirectional associative memories (BAM), the Hopfield model, continuous Hopfield model, simulated annealing, the Boltzmann machine.
  • Hopfield networks in the search for suboptimal solutions of NP-complete problems.

5. Self-organization and hybrid models

  • Unsupervised reinforcement learning - Oja´s algorithm for PCA.
  • Kohonen self-organizing feature maps and algorithms for their training, lateral inhibition, topological neighborhood.
  • Counter-propagation neural networks, RBF-networks, Adaptive Resonance Theory (ART).
  • Cascade correlation and modular neural networks - mixtures of local experts.

6. Genetic algorithms

  • Coding of the optimization problem, population of strings, fundamental genetic operators - selection, cross-over, mutation.
  • Fitness function. Convergence analysis - schemata theorem.
  • Applications of genetic algorithms in the field of artificial neural networks.

 
Charles University | Information system of Charles University | http://www.cuni.cz/UKEN-329.html