Squeezing bottlenecks: Exploring the limits of autoencoder semantic representation capabilities

NEUROCOMPUTING

卷:175

Article; Proceedings Paper

Gupta, Parth¹  Banchs, Rafael E.²  Rosso, Paolo¹ 

[1] Univ Politecn Valencia, PRHLT Res Ctr, E-46022 Valencia, Spain

[2] Inst Infocomm Res, Singapore, Singapore

关键词: Text representation;    Deep autoencoder;   

DOI  :  10.1016/j.neucom.2015.06.091

来源: Elsevier

【 摘 要 】

We present a comprehensive study on the use of autoencoders for modelling text data, in which (differently from previous studies) we focus our attention on the various issues. We explore the suitability of two different models binary deep autencoders (bDA) and replicated-softmax deep autencoders (rsDA) for constructing deep autoencoders for text data at the sentence level. We propose and evaluate two novel metrics for better assessing the text-reconstruction capabilities of autoencoders. We propose an automatic method to find the critical bottleneck dimensionality for text representations (below which structural information is lost); and finally we conduct a comparative evaluation across different languages, exploring the regions of critical bottleneck dimensionality and its relationship to language perplexity. (C) 2015 Elsevier B.V. All rights reserved.

【 授权许可】

Free