Linguistic Issues in Language Technology | |
Can Recurrent Neural Networks Learn Nested Recursion? | |
Jean-Philippe Bernardy1  | |
[1] University of Gothenburg | |
关键词: RNN; CFG; nested recursion; machine learning; grammar; | |
DOI : | |
学科分类:社会科学、人文和艺术(综合) | |
来源: C S L I Publications | |
【 摘 要 】
Context-free grammars (CFG) were one of the first formal tools used to model natural languages, and they remain relevant today as the basis of several frameworks. A key ingredient of CFG is the presence of nested recursion.In this paper, we investigate experimentally the capability of sev- eral recurrent neural networks (RNNs) to learn nested recursion. More precisely, we measure an upper bound of their capability to do so, by simplifying the task to learning a generalized Dyck language, namely one composed of matching parentheses of various kinds. To do so, we present the RNNs with a set of random strings having a given maximum nesting depth and test its ability to predict the kind of closing parenthesis when facing deeper nested strings. We report mixed results when generalizing to deeper nesting levels, the accuracy of standard RNNs is significantly higher than random, but still far from perfect. Additionally, we propose some non-standard stack-based models which can approach perfect accuracy, at the cost of robustness.
【 授权许可】
Unknown
【 预 览 】
Files | Size | Format | View |
---|---|---|---|
RO201901233404207ZK.pdf | 304KB | download |