Scheduling multicore workload on shared multipurpose clusters

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Templon, J.A.^1 ; Acosta-Silva, C.^2,4 ; Flix Molina, J.^2,3 ; Forti, A.C.^5 ; Yzquierdo, A Pérez-Calero^2,3 ; Starink, R.^1

Nikhef, Science Park 105, XG Amsterdam, NL

1098, Netherlands^1

Port d'Informació Científica (PIC), Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain^2

Also Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas (CIEMAT), Madrid, Spain^3

Institut de Física d'Altes Energies, IFAE, Edifici Cn, Universitat Autònoma de Barcelona, Bellaterra (Barcelona), Spain^4

School of Physics and Astronomy, University of Manchester, Oxford Road, Manchester

M13 9PL, United Kingdom^5

关键词: Batch schedulers;    Batch systems;    Computing sites;    Job size;    Multi core;    Ramp up;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/5/052038/pdf
DOI  :  10.1088/1742-6596/664/5/052038

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

With the advent of workloads containing explicit requests for multiple cores in a single grid job, grid sites faced a new set of challenges in workload scheduling. The most common batch schedulers deployed at HEP computing sites do a poor job at multicore scheduling when using only the native capabilities of those schedulers. This paper describes how efficient multicore scheduling was achieved at the sites the authors represent, by implementing dynamically-sized multicore partitions via a minimalistic addition to the Torque/Maui batch system already in use at those sites. The paper further includes example results from use of the system in production, as well as measurements on the dependence of performance (especially the ramp-up in throughput for multicore jobs) on node size and job size.