Extracting the Pairs of Opinion Target and Opinion Term From Reviews With Adaptive Crowd Labeling

IEEE Access

You Li¹  Wei Zhao²  Ya Zhou²  Yuming Lin²  Huibing Zhang² 

[1] Guangxi Key Laboratory of Automatic Detecting Technology and Instruments, Guilin University of Electronic Technology, Guilin, China;Guangxi Key Laboratory of Trusted Software, Guilin University of Electronic Technology, Guilin, China;

关键词: Crowdsourcing;    opinion target;    opinion term;    worker assessment;    budget constraint;   

DOI  :  10.1109/ACCESS.2019.2930905

来源: DOAJ

【 摘 要 】

Labeling samples manually is a laborious, error-prone, and cost-consuming process, which is a traditional approach to preparing training data for supervised learning. Crowdsourcing provides an effective way to acquire labeled training data. In this paper, we propose an adaptive crowd labeling method to construct a set of the pairs of opinion target and opinion term iteratively under certain budget constraint. First, we assess the workers' reliabilities with a small number of labeled samples based on an EM process, since the worker's expertise varies widely on an open crowdsourcing platform. And then, the tasks are assigned to the high reliable workers for labeling the pairs of opinion target and opinion term. Finally, the responds of a sample from multiple workers are integrated to generate a final result based on the labelers' reliabilities as well as the dependency relation between opinion target and opinion term in this sample. A series of the experimental results show that the proposed method can achieve better extracting effectiveness compared with the baselines and the state-of-the-art methods.

【 授权许可】

Unknown