New and Efficient Algorithms for Producing Frequent Itemsets with the Map-Reduce Framework

Algorithms

Yaron Gonen¹  Ehud Gudes¹  Kirill Kandalov² 

[1] Department of Computer Science, Ben-Gurion University, Beer-Sheva 8410501, Israel;Department of Computer Science, Open University, Ra’anana 4353701, Israel;

关键词: apriori;    map reduce;    big data;    frequent itemsets;    closed itemsets;    incremental computation;   

DOI  :  10.3390/a11120194

来源: DOAJ

【 摘 要 】

The Map-Reduce (MR) framework has become a popular framework for developing new parallel algorithms for Big Data. Efficient algorithms for data mining of big data and distributed databases has become an important problem. In this paper we focus on algorithms producing association rules and frequent itemsets. After reviewing the most recent algorithms that perform this task within the MR framework, we present two new algorithms: one algorithm for producing closed frequent itemsets, and the second one for producing frequent itemsets when the database is updated and new data is added to the old database. Both algorithms include novel optimizations which are suitable to the MR framework, as well as to other parallel architectures. A detailed experimental evaluation shows the effectiveness and advantages of the algorithms over existing methods when it comes to large distributed databases.

【 授权许可】

Unknown