Evolution of CMS workload management towards multicore job support

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Pérez-Calero Yzquierdo, A.^1,2 ; Hernández, J.M.^2 ; Khan, F.A.^3 ; Letts, J.^4 ; Majewski, K.^5 ; Rodrigues, A.M.^5 ; McCrea, A.^4 ; Vaandering, E.^5

Port d'Informació Cientifica (PIC), Universitat Autónoma de Barcelona, Bellaterra (Barcelona), Spain^1

Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas, CIEMAT, Madrid, Spain^2

National Centre for Physics, Quaid-i-Azam University, Pakistan^3

University of California San Diego, United States^4

Fermi National Accelerator Laboratory, United States^5

关键词: Data reconstruction;    Distributed computing systems;    Distributed resources;    Multi-core processor architectures;    Resource provisioning;    Scheduling strategies;    Sequential programming;    Simulation algorithms;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/6/062046/pdf
DOI  :  10.1088/1742-6596/664/6/062046

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The successful exploitation of multicore processor architectures is a key element of the LHC distributed computing system in the coming era of the LHC Run 2. High-pileup complex-collision events represent a challenge for the traditional sequential programming in terms of memory and processing time budget. The CMS data production and processing framework is introducing the parallel execution of the reconstruction and simulation algorithms to overcome these limitations. CMS plans to execute multicore jobs while still supporting singlecore processing for other tasks difficult to parallelize, such as user analysis. The CMS strategy for job management thus aims at integrating single and multicore job scheduling across the Grid. This is accomplished by employing multicore pilots with internal dynamic partitioning of the allocated resources, capable of running payloads of various core counts simultaneously. An extensive test programme has been conducted to enable multicore scheduling with the various local batch systems available at CMS sites, with the focus on the Tier-0 and Tier-1s, responsible during 2015 of the prompt data reconstruction. Scale tests have been run to analyse the performance of this scheduling strategy and ensure an efficient use of the distributed resources. This paper presents the evolution of the CMS job management and resource provisioning systems in order to support this hybrid scheduling model, as well as its deployment and performance tests, which will enable CMS to transition to a multicore production model for the second LHC run.