Story Generation Using Dynamic Text Infilling
| Název práce v češtině: | Generování příběhů pomocí dynamického rozšiřování textu |
|---|---|
| Název v anglickém jazyce: | Story Generation Using Dynamic Text Infilling |
| Klíčová slova: | generování příběhů|jazykové modely|neuronové sítě |
| Klíčová slova anglicky: | story generation|language models|neural networks |
| Akademický rok vypsání: | 2022/2023 |
| Typ práce: | bakalářská práce |
| Jazyk práce: | angličtina |
| Ústav: | Ústav formální a aplikované lingvistiky (32-UFAL) |
| Vedoucí / školitel: | Ing. Zdeněk Kasner, Ph.D. |
| Řešitel: | Bc. Katarína Bucková - zadáno a potvrzeno stud. odd. |
| Datum přihlášení: | 01.11.2022 |
| Datum zadání: | 02.11.2022 |
| Datum potvrzení stud. oddělením: | 23.11.2022 |
| Datum a čas obhajoby: | 07.09.2023 09:00 |
| Datum odevzdání elektronické podoby: | 20.07.2023 |
| Datum odevzdání tištěné podoby: | 20.07.2023 |
| Datum proběhlé obhajoby: | 07.09.2023 |
| Oponenti: | Mgr. Rudolf Rosa, Ph.D. |
| Zásady pro vypracování |
| Currently, it is possible to generate long-form stories using Transformer-based pretrained neural language models such as GPT-2 (Radford et al., 2019). However, it is hard to guarantee the logical coherency of the story. Even if the story is generated hierarchically (Schmidtová, 2022), the model may diverge from the outline due to the nature of left-to-right autoregressive decoding, which provides only a weak supervision signal.
An alternative to left-to-right decoding is text infilling, i.e., generating a missing span of variable length given its left and right context (Donahue et al., 2020). As shown by Donahue et al. (2020), this technique can be used to infill masked parts of short stories from the ROCStories dataset (Mostafazadeh et al., 2016). The goal of the work is to examine the potential of text infilling for generating longer stories. The student will (1) finetune a pretrained neural language model for text infilling, (2) implement a system that will recursively apply the model to dynamically expand the story outline, and (3) evaluate the quality of the system outputs using both automatic metrics and human-based ratings. For stopping the recursion, the system should identify which parts of the story can be expanded while keeping the story coherent (Mori et al., 2020). |
| Seznam odborné literatury |
| Donahue, C., Lee, M., & Liang, P. (2020, July). Enabling Language Models to Fill in the Blanks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 2492-2501).
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language Models are Unsupervised Multitask Learners,” Technical Report, OpenAI, Feb. 2019. https://openai.com/blog/better-language-models/ Mostafazadeh, N., Chambers, N., He, X., Parikh, D., Batra, D., Vanderwende, L., ... & Allen, J. (2016, June). A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of NAACL HLT 2016 (pp. 839-849). Mori, Y., Yamane, H., Mukuta, Y., & Harada, T. (2020). Finding and generating a missing part for story completion. In Proceedings of the The 4th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (pp. 156-166). Schmidtová, P. (2022). Theatre play generation. [Master Thesis, Charles University]. |