Výzkum reprezentací v hlubokém učení při zpracování jazyka
| Thesis title in Czech: | Výzkum reprezentací v hlubokém učení při zpracování jazyka |
|---|---|
| Thesis title in English: | Exploring Deep Learning Representations in NLP |
| Academic year of topic announcement: | 2018/2019 |
| Thesis type: | dissertation |
| Thesis language: | |
| Department: | Institute of Formal and Applied Linguistics (32-UFAL) |
| Supervisor: | doc. RNDr. Ondřej Bojar, Ph.D. |
| Author: | hidden - assigned and confirmed by the Study Dept. |
| Date of registration: | 16.07.2019 |
| Date of assignment: | 16.07.2019 |
| Confirmed by Study dept. on: | 04.10.2019 |
| Guidelines |
| Deep neural networks (DNN) have become the standard means for many complex tasks of natural language processing (NLP). They can very effectively find relevant patterns in the data and learn to mimic human behaviour even in situations where no clear rules can be explicitly stated. This strength of DNNs comes from their ability to implicitly learn continuous-space representations of the inputs, at many representation levels at once. The nature of these representations is not generally well understood for the majority of language processing tasks and their relevance to natural representations in the brains of language users is at best speculative.
The goal of the thesis is to study the internal representations that are learnt automatically by DNNs, using established DNN exploration techniques on networks trained for NLP tasks and proposing novel exploration methods. The thesis will focus on and gradually progress over a range of NLP tasks, from simpler ones like language identification over word or text classification to complex ones like machine translation. Where possible, the thesis would search for potential parallels of the handling of the task in DNNs and in humans, proposing various psycholinguistic experiments, e.g. self-paced reading or eye-tracking studies, to validate or reject similarities between human and artificial representations and processing. |
| References |
| Goodfellow, I., Y. Bengio, and A. Courville 2016. Deep learning. Cambridge, MA, USA: MIT press.
Petr Sgall, Eva Hajičová a Jarmila Panevová. The Meaning of the Sentence and Its Semantic and Pragmatic Aspects. Academia/Reidel Publishing Company, Praha/Dordrecht, 1986. Bates, Elizabeth. Processing complex sentences: A cross-linguistic study. Language and cognitive processes 14.1 (1999): 69-123. Raghu, Maithra, et al. On the expressive power of deep neural networks. Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017. Guillaume Alain, Yoshua Bengio. Understanding intermediate layers using linear classifier probes. ICLR (Workshop) 2017. |