Hardware trends over the last several decades have lead to shifting prioritieswith respect to performance bottlenecks in the implementations of dataflowstypically present in large-scale data analytics applications. In particular,efficient use of main memory has emerged as a critical aspect of dataflowimplementation, due to the proliferation of multi-core architectures, as well asthe rapid development of faster-than-disk storage media. At the same time, thewealth of static domain-specific information about applications remains anuntapped resource when it comes to optimizing the use of memory in a dataflowapplication.We propose a compilation-based approach to the synthesis of memory-efficientdataflow implementations, using static analysis to extract and leveragedomain-specific information about the application. Our program transformationsuse the combined results of type, effect, and provenance analyses to infer time-and space- effective placement of primitive memory operations, precluding theneed for dynamic memory management and its attendant costs. The experimentalevaluation of implementations synthesized with our framework shows both theimportance of optimizing for memory performance, as well as significant benefitsof our approach, along multiple dimensions.Finally, we also demonstrate a framework for formally verifying the soundness ofthese transformations, laying the foundation for their use as a component of amore general implementation synthesis ecosystem.
【 预 览 】
附件列表
Files
Size
Format
View
Domain Specific Memory Management for Large Scale Data Analytics
PDF
download
878987797
028-85220240
