An Approach of RDD Optimization in Big Data Analytics

International Conference on Recent Advancements and Effectual Researches in Engineering Science and Technology

Athira, V.^1 ; Thomas, Jeena^1

Department of Computer Science, St. Josephs College of Engineering and Technology, Palai, Kerala, India^1

关键词: Analysis frameworks;    Big Data Analytics;    Computing paradigm;    Distributed data analysis;    Information age;    Point moves;    Resilient distributed dataset;    Technological change;   

Others  :  https://iopscience.iop.org/article/10.1088/1757-899X/396/1/012022/pdf
DOI  :  10.1088/1757-899X/396/1/012022

来源: IOP

【 摘 要 】

We live in the information age and new information is produced in each and every second. The information can be worlds data which is responsible for major technological changes that can bring new ways in decision making. Keeping this data for further analysis and computation is a difficult task. Several studies are done in this arena, making it effective for future figuring. Processing or analysing such huge amount of data is a challenging task. All the existing technologies contain certain bottlenecks and performance overheads, diverse challenges like scalability. Spark is the commonly used data analysis framework. Map Reduce is a computing paradigm and a popular model for distributed data analysis. This paper gives a survey about some enormous information technologies, how it will deal with huge data, and the difficulties in existing advances, and has additionally learned about a portion of the execution bottlenecks and preventive techniques, and the concentration at that point moves to the Resilient Distributed Dataset (RDD), and how it is optimized.