Automatická analýza výsledků benchmarků pomocí strojového účení
| Thesis title in Czech: | Automatická analýza výsledků benchmarků pomocí strojového účení |
|---|---|
| Thesis title in English: | Automating Benchmark Analysis Through Machine Learning |
| English key words: | machine learning|software benchmarking|automated analysis |
| Academic year of topic announcement: | 2024/2025 |
| Thesis type: | Bachelor's thesis |
| Thesis language: | |
| Department: | Department of Distributed and Dependable Systems (32-KDSS) |
| Supervisor: | Mgr. Vojtěch Horký, Ph.D. |
| Author: | hidden - assigned and confirmed by the Study Dept. |
| Date of registration: | 04.09.2024 |
| Date of assignment: | 04.09.2024 |
| Confirmed by Study dept. on: | 04.09.2024 |
| Guidelines |
| Regular software performance benchmarking is employed when there is a need to prevent silent performance deterioration of the software product. Automating the benchmark execution is easy but the challenges are in interpretation of the data: software rarely behaves in a fully deterministic manner and sophisticated methods must be used to detect performance changes.
The goal of this thesis is to employ methods of machine learning to detect software performance changes. Unlike strict statistical interpretation of the results we believe that machine learning methods can better "learn" what represents a true performance change that should be reported. The focus of the thesis should be on automating the processing of benchmark results as to provide users (such as performance engineers) only with cases requiring further manual inspection and screening them from checking irrelevant changes manually. The resulting pipeline built on machine learning processes (probably both supervised and unsupervised learning methods would need to be employed) should be able to read historical measurements and use them to analyze benchmark results and report anomalies (i.e., performance changes). Tentative reviewers: P. Tůma, L. Bulej |
| References |
| - Saiqa, Aleem., Luiz, Fernando, Capretz., Faheem, Ahmed. (2015). Benchmarking Machine Learning Technologies for Software Defect Detection.
- Sudipto, Nandan. (2020). Performance Benchmark using Machine Learning. - Samuel Kounev, Klaus-Dieter Lange, Jóakim von Kistowski (202), Systems Benchmarking: For Scientists and Engineers. |