Performance Evaluation of an Independent Time Optimized Infrastructure for Big Data Analytics that Maintains Symmetry

Symmetry

Satvik Vats¹  BharatBhushan Sagar¹  BrunoA. Pansera²  Ali Ahmadian³  Karan Singh⁴ 

[1] Computer Science and Engineering, Birla Institute of Technology, Mesra, Ranchi 835215, India;DiGiES & Decisions Lab, Mediterranea University of Reggio Calabria, 89124 Reggio Calabria, Italy;Institute of Industry Revolution 4.0, The National University of Malaysia, Bangi, UKM 43600, Malaysia;School of Computer & Systems Sciences, Jawaharlal Nehru University, New Delhi 110067, India;

关键词: hadoop;    MapReduce;    big data;    response time;    symmetrical streaming;    machine Learning;   

DOI  :  10.3390/sym12081274

来源: DOAJ

【 摘 要 】

Traditional data analytics tools are designed to deal with the asymmetrical type of data i.e., structured, semi-structured, and unstructured. The diverse behavior of data produced by different sources requires the selection of suitable tools. The restriction of recourses to deal with a huge volume of data is a challenge for these tools, which affects the performances of the tool’s execution time. Therefore, in the present paper, we proposed a time optimization model, shares common HDFS (Hadoop Distributed File System) between three Name-node (Master Node), three Data-node, and one Client-node. These nodes work under the DeMilitarized zone (DMZ) to maintain symmetry. Machine learning jobs are explored from an independent platform to realize this model. In the first node (Name-node 1), Mahout is installed with all machine learning libraries through the maven repositories. The second node (Name-node 2), R connected to Hadoop, is running through the shiny-server. Splunk is configured in the third node (Name-node 3) and is used to analyze the logs. Experiments are performed between the proposed and legacy model to evaluate the response time, execution time, and throughput. K-means clustering, Navies Bayes, and recommender algorithms are run on three different data sets, i.e., movie rating, newsgroup, and Spam SMS data set, representing structured, semi-structured, and unstructured data, respectively. The selection of tools defines data independence, e.g., Newsgroup data set to run on Mahout as others cannot be compatible with this data. It is evident from the outcome of the data that the performance of the proposed model establishes the hypothesis that our model overcomes the limitation of the resources of the legacy model. In addition, the proposed model can process any kind of algorithm on different sets of data, which resides in its native formats.

【 授权许可】

Unknown