Effective Transfer Learning with Label-Based Discriminative Feature Learning

Sensors

Sangwoo Kang¹  Gyunyeop Kim¹ 

[1] School of Computing, Gachon University, Seongnam 13120, Korea;

关键词: natural language processing;    transfer learning;    pre-training;    word embedding;   

DOI  :  10.3390/s22052025

来源: DOAJ

【 摘 要 】

The performance of natural language processing with a transfer learning methodology has improved by applying pre-training language models to downstream tasks with a large number of general data. However, because the data used in pre-training are irrelevant to the downstream tasks, a problem occurs in that it learns general features rather than those features specific to the downstream tasks. In this paper, a novel learning method is proposed for embedding pre-trained models to learn specific features of such tasks. The proposed method learns the label features of downstream tasks through contrast learning using label embedding and sampled data pairs. To demonstrate the performance of the proposed method, we conducted experiments on sentence classification datasets and evaluated whether the features of the downstream tasks have been learned through a PCA and a clustering of the embeddings.

【 授权许可】

Unknown