Transformer Architectures for Multi-Channel Data
| Thesis title in Czech: | Architektury Transformerů pro Vícekanálová Data |
|---|---|
| Thesis title in English: | Transformer Architectures for Multi-Channel Data |
| Key words: | Strojové učení|Vícekanálové transformery|Biometrická data |
| English key words: | Machine Learning|Multi-channel Transformers|Biometric data |
| Academic year of topic announcement: | 2023/2024 |
| Thesis type: | Bachelor's thesis |
| Thesis language: | angličtina |
| Department: | Institute of Formal and Applied Linguistics (32-UFAL) |
| Supervisor: | doc. RNDr. Ondřej Bojar, Ph.D. |
| Author: | Bc. Ilia Zavidnyi - assigned and confirmed by the Study Dept. |
| Date of registration: | 20.12.2023 |
| Date of assignment: | 20.12.2023 |
| Confirmed by Study dept. on: | 24.01.2024 |
| Date and time of defence: | 28.06.2024 09:00 |
| Date of electronic submission: | 09.05.2024 |
| Date of submission of printed version: | 09.05.2024 |
| Date of proceeded defence: | 28.06.2024 |
| Opponents: | Mgr. Peter Polák |
| Guidelines |
| The aim of the thesis is to explore and evaluate the effectiveness of Transformer architectures in the context of multi-channel data classification or regression tasks, with a specific focus on biometric datasets like MIMIC. The widespread adoption of Transformer models in natural language processing, computer vision, and other sequence processing or prediction tasks has shown their capacity to capture intricate dependencies in sequential data, making them a promising candidate for processing multi-channel data from various sources simultaneously.
The thesis will focus on the following objectives: 1. Design and Implementation of Multi-channel Transformer Architecture: Develop a specialized Transformer architecture tailored for handling multi-channel data. Considerations will be made for accommodating the unique characteristics of biometric datasets, such as temporal dependencies, varying signal modalities (numerical, categorical), highly varied sampling rates across channels, and missing data points. 2. Empirical Evaluation: Conduct a comprehensive evaluation of the proposed Transformer architecture on biometric datasets, such as MIMIC. Assess the model’s accuracy, efficiency, and scalability compared to existing methods. We will evaluate several setups of the model on one or two machine learning tasks (regression or classification). Examples of such tasks include, but are not limited to: sleep apnea prediction or classification of a diagnosis coded using ICD ontology. |
| References |
| Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L. A., & Mark, R. (2021). MIMIC-IV (version 1.0). PhysioNet. https://doi.org/10.13026/s6n6-xd98.
Feng-Ju Chang, Martin Radfar, Athanasios Mouchtaris, Brian King, & Siegfried Kunzmann. (2021). End-to-End Multi-Channel Transformer for Speech Recognition. https://arxiv.org/abs/2102.03951 Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, & Artem Babenko. (2023). Revisiting Deep Learning Models for Tabular Data. https://arxiv.org/pdf/2106.11959.pdf Xie F, Zhou J, Lee JW, Tan M, Li SQ, Rajnthern L, Chee ML, Chakraborty B, Wong AKI, Dagan A, Ong MEH, Gao F, Liu N. Benchmarking emergency department prediction models with machine learning and public electronic health records. Scientific Data 2022 Oct; 9: 658. https://doi.org/10.1038/s41597-022-01782-9 |