Výzkum reprezentací v hlubokém učení při zpracování jazyka
| Název práce v češtině: | Výzkum reprezentací v hlubokém učení při zpracování jazyka |
|---|---|
| Název v anglickém jazyce: | Exploring Deep Learning Representations in NLP |
| Akademický rok vypsání: | 2018/2019 |
| Typ práce: | disertační práce |
| Jazyk práce: | |
| Ústav: | Ústav formální a aplikované lingvistiky (32-UFAL) |
| Vedoucí / školitel: | doc. RNDr. Ondřej Bojar, Ph.D. |
| Řešitel: | skrytý - zadáno a potvrzeno stud. odd. |
| Datum přihlášení: | 16.07.2019 |
| Datum zadání: | 16.07.2019 |
| Datum potvrzení stud. oddělením: | 04.10.2019 |
| Zásady pro vypracování |
| Deep neural networks (DNN) have become the standard means for many complex tasks of natural language processing (NLP). They can very effectively find relevant patterns in the data and learn to mimic human behaviour even in situations where no clear rules can be explicitly stated. This strength of DNNs comes from their ability to implicitly learn continuous-space representations of the inputs, at many representation levels at once. The nature of these representations is not generally well understood for the majority of language processing tasks and their relevance to natural representations in the brains of language users is at best speculative.
The goal of the thesis is to study the internal representations that are learnt automatically by DNNs, using established DNN exploration techniques on networks trained for NLP tasks and proposing novel exploration methods. The thesis will focus on and gradually progress over a range of NLP tasks, from simpler ones like language identification over word or text classification to complex ones like machine translation. Where possible, the thesis would search for potential parallels of the handling of the task in DNNs and in humans, proposing various psycholinguistic experiments, e.g. self-paced reading or eye-tracking studies, to validate or reject similarities between human and artificial representations and processing. |
| Seznam odborné literatury |
| Goodfellow, I., Y. Bengio, and A. Courville 2016. Deep learning. Cambridge, MA, USA: MIT press.
Petr Sgall, Eva Hajičová a Jarmila Panevová. The Meaning of the Sentence and Its Semantic and Pragmatic Aspects. Academia/Reidel Publishing Company, Praha/Dordrecht, 1986. Bates, Elizabeth. Processing complex sentences: A cross-linguistic study. Language and cognitive processes 14.1 (1999): 69-123. Raghu, Maithra, et al. On the expressive power of deep neural networks. Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017. Guillaume Alain, Yoshua Bengio. Understanding intermediate layers using linear classifier probes. ICLR (Workshop) 2017. |