Running ATLAS workloads within massively parallel distributed applications using Athena Multi-Process framework (AthenaMP)

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Calafiura, Paolo^1 ; Leggett, Charles^1 ; Seuster, Rolf^2 ; Tsulaia, Vakhtang^1 ; Van Gemmeren, Peter^3

Lawrence Berkeley National Laboratory, 1 Cyclotron Rd, Berkeley

CA

94720, United States^1

TRIUMF, 4004 Wesbrook Mall, Vancouver

BC

2A3, Canada^2

Argonne National Laboratory, 9700 S. Cass Ave, Argonne

IL

60439, United States^3

关键词: Analysis frameworks;    Application codes;    Computing resource;    Event Processing;    Event processors;    Massively parallels;    Memory footprint;    Multi-Processes;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/7/072050/pdf
DOI  :  10.1088/1742-6596/664/7/072050

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

AthenaMP is a multi-process version of the ATLAS reconstruction, simulation and data analysis framework Athena. By leveraging Linux fork and copy-on-write mechanisms, it allows for sharing of memory pages between event processors running on the same compute node with little to no change in the application code. Originally targeted to optimize the memory footprint of reconstruction jobs, AthenaMP has demonstrated that it can reduce the memory usage of certain configurations of ATLAS production jobs by a factor of 2. AthenaMP has also evolved to become the parallel event-processing core of the recently developed ATLAS infrastructure for fine-grained event processing (Event Service) which allows the running of AthenaMP inside massively parallel distributed applications on hundreds of compute nodes simultaneously. We present the architecture of AthenaMP, various strategies implemented by AthenaMP for scheduling workload to worker processes (for example: Shared Event Queue and Shared Distributor of Event Tokens) and the usage of AthenaMP in the diversity of ATLAS event processing workloads on various computing resources: Grid, opportunistic resources and HPC.