Effective Parallel Processing Social Media Analytics Framework

Journal of King Saud University: Computer and Information Sciences

Harsh Kumar Verma¹  Ravindra Kumar Singh² 

[1] Department of Computer Science and Engineering, Dr. B. R. Ambedkar National Institute of Technology, Jalandhar, India;Corresponding author.;

关键词: Social media analytics;    Real time analytics;    MongoDB;    Redis;    Python-dash;    Visualization;   

DOI  :  

来源: DOAJ

【 摘 要 】

The widespread adoption of opinion mining and sentiment analysis in higher cognitive processes encourages the need for real-time processing of social media data to capture insights about user's sentiment polarity, user’s opinion, and current trends of the domain. In recent years lots of research were conducted and various machine learning algorithms were developed around the processing of data to achieve higher accuracy while reducing the processing time is still challenging. Big data technologies came unraveled these challenges but they have their own set of complexities along with having hardware deadweight on the system. The contribution of this research paper is to touch upon the mentioned challenges by presenting a climbable, instantaneous and fault-tolerant framework to process real-time data to extract hidden insights within it while not bearing any additional overhead of big data technologies. This framework is versatile enough to support batch processing along with real-time data streams in parallel and distributed environments. Experimental results concluded that 4-threaded parallel architecture of the framework performs at 2X speed compared to single-threaded architecture and shared URLs, embedded Images and Author's meta info it boosting the tweets prediction. Moreover, this research additionally provides a comparison of Support Vector Machines (SVM), Light GBM (LGBM) and Long Short Term Memory (LSTM) supervised machine learning techniques for sentiment analysis and concluded LGBM is the most effective model.

【 授权许可】

Unknown