Transformer Architectures for Multi-Channel Data
| Název práce v češtině: | Architektury Transformerů pro Vícekanálová Data |
|---|---|
| Název v anglickém jazyce: | Transformer Architectures for Multi-Channel Data |
| Klíčová slova: | Strojové učení|Vícekanálové transformery|Biometrická data |
| Klíčová slova anglicky: | Machine Learning|Multi-channel Transformers|Biometric data |
| Akademický rok vypsání: | 2023/2024 |
| Typ práce: | bakalářská práce |
| Jazyk práce: | angličtina |
| Ústav: | Ústav formální a aplikované lingvistiky (32-UFAL) |
| Vedoucí / školitel: | doc. RNDr. Ondřej Bojar, Ph.D. |
| Řešitel: | Bc. Ilia Zavidnyi - zadáno a potvrzeno stud. odd. |
| Datum přihlášení: | 20.12.2023 |
| Datum zadání: | 20.12.2023 |
| Datum potvrzení stud. oddělením: | 24.01.2024 |
| Datum a čas obhajoby: | 28.06.2024 09:00 |
| Datum odevzdání elektronické podoby: | 09.05.2024 |
| Datum odevzdání tištěné podoby: | 09.05.2024 |
| Datum proběhlé obhajoby: | 28.06.2024 |
| Oponenti: | Mgr. Peter Polák |
| Zásady pro vypracování |
| The aim of the thesis is to explore and evaluate the effectiveness of Transformer architectures in the context of multi-channel data classification or regression tasks, with a specific focus on biometric datasets like MIMIC. The widespread adoption of Transformer models in natural language processing, computer vision, and other sequence processing or prediction tasks has shown their capacity to capture intricate dependencies in sequential data, making them a promising candidate for processing multi-channel data from various sources simultaneously.
The thesis will focus on the following objectives: 1. Design and Implementation of Multi-channel Transformer Architecture: Develop a specialized Transformer architecture tailored for handling multi-channel data. Considerations will be made for accommodating the unique characteristics of biometric datasets, such as temporal dependencies, varying signal modalities (numerical, categorical), highly varied sampling rates across channels, and missing data points. 2. Empirical Evaluation: Conduct a comprehensive evaluation of the proposed Transformer architecture on biometric datasets, such as MIMIC. Assess the model’s accuracy, efficiency, and scalability compared to existing methods. We will evaluate several setups of the model on one or two machine learning tasks (regression or classification). Examples of such tasks include, but are not limited to: sleep apnea prediction or classification of a diagnosis coded using ICD ontology. |
| Seznam odborné literatury |
| Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L. A., & Mark, R. (2021). MIMIC-IV (version 1.0). PhysioNet. https://doi.org/10.13026/s6n6-xd98.
Feng-Ju Chang, Martin Radfar, Athanasios Mouchtaris, Brian King, & Siegfried Kunzmann. (2021). End-to-End Multi-Channel Transformer for Speech Recognition. https://arxiv.org/abs/2102.03951 Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, & Artem Babenko. (2023). Revisiting Deep Learning Models for Tabular Data. https://arxiv.org/pdf/2106.11959.pdf Xie F, Zhou J, Lee JW, Tan M, Li SQ, Rajnthern L, Chee ML, Chakraborty B, Wong AKI, Dagan A, Ong MEH, Gao F, Liu N. Benchmarking emergency department prediction models with machine learning and public electronic health records. Scientific Data 2022 Oct; 9: 658. https://doi.org/10.1038/s41597-022-01782-9 |