Optimalizace a náhradní modely v automatickém strojovém učení

• Thesis title in Czech: Optimalizace a náhradní modely v automatickém strojovém učení
• Thesis title in English: Optimization and surrogate models in AutoML
• Academic year of topic announcement: 2021/2022
• Thesis type: dissertation
• Thesis language: čeština
• Department: Department of Theoretical Computer Science and Mathematical Logic (32-KTIML)
• Supervisor: Mgr. Roman Neruda, CSc.
• Author: hidden - assigned and confirmed by the Study Dept.
• Date of registration: 20.09.2021
• Date of assignment: 20.09.2021
• Confirmed by Study dept. on: 21.09.2021
• Advisors: RNDr. Věra Kůrková, DrSc.

Guidelines

The field of automated machine learning (or AutoML) is a rapidly evolving research area that aims to answer problems across different domains such as automatic data collection and preprocessing, model selection or hyperparameter tuning. Recently, promising advances have been made in the subfield of AutoML - neural architecture search (or NAS) - where the task is to find the best-performing architecture for a given machine learning problem.

The goal of the thesis is to develop AutoML or NAS systems using new approaches of solving the problem, as well as to explore some of the theoretical underpinnings of the fields. Reproducibility and comparison with existing systems is a key step of AutoML and NAS research that has been often an issue, as many state-of-the-art systems had a similar performance as baseline random search or were heavily influenced by the random seed.
The search space and a corresponding optimization method form another factor that has a high impact on the results of a particular NAS problem. It is on open problem, if a global search space is better than modular search spaces or not.
Another challenge is the performance prediction of models, which can speed up the search process. Adaptive surrogate models estimating the performance of solution candidates will be investigated. A promising approach is based on graph embeddings of solution schemes by means of a tailored graph neural network.

References

1. Ian J. Goodfellow, Yoshua Bengio and Aaron Courville: Deep Learning, MIT Press, 2016.

2. Frank Hutter, Lars Kotthoff, Joaquin Vanschoren (Eds): Automated Machine Learning - Methods, Systems, Challenges. Springer, 2019.

3. Xin He, Kaiyong Zhao, Xiaowen Chu: AutoML: A survey of the state-of-the-art, Knowledge-Based Systems, vol. 212, Elsevier, 2021.

4. Martin Wistuba, Ambrish Rawat, Tejaswini Pedapati: A Survey on Neural Architecture Search, arXiv: 1905.01392, 2019.

5. Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po-Yao Huang, Zhihui Li, Xiaojiang Chen, Xin Wang: A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions. arXiv:2006.02903, 2021.

6. Antoine Yang, Pedro M. Esperança, Fabio M. Carlucci: NAS evaluation is frustratingly hard, Proceedings of The International Conference on Learning Representations (ICLR), 2020.

7. Liu, Chenxi and Zoph, Barret and Neumann, Maxim and Shlens, Jonathon and Hua, Wei and Li, Li-Jia and Fei-Fei, Li and Yuille, Alan and Huang, Jonathan and Murphy, Kevin: Progressive Neural Architecture Search, Proceedings of the European Conference on Computer Vision (ECCV), 2018.