Dynamic Resource Allocation with the arcControlTower

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Filipi, A.^1 ; Cameron, D.^2 ; Nilsen, J.K.^2

Jozef Stefan Institute, Jamova 39, Ljubljana

1000, Slovenia^1

University of Oslo, P.b. 1048 Blindern, Oslo

N-0316, Norway^2

关键词: Computing element;    Computing technology;    Distributed computing resources;    Dynamic resource allocations;    Job description;    Job management;    Job management system;    Nordic countries;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/6/062015/pdf
DOI  :  10.1088/1742-6596/664/6/062015

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

Distributed computing resources available for high-energy physics research are becoming less dedicated to one type of workflow and researchers workloads are increasingly exploiting modern computing technologies such as parallelism. The current pilot job management model used by many experiments relies on static dedicated resources and cannot easily adapt to these changes. The model used for ATLAS in Nordic countries and some other places enables a flexible job management system based on dynamic resources allocation. Rather than a fixed set of resources managed centrally, the model allows resources to be requested on the fly. The ARC Computing Element (ARC-CE) and ARC Control Tower (aCT) are the key components of the model. The aCT requests jobs from the ATLAS job management system (PanDA) and submits a fully-formed job description to ARC-CEs. ARC-CE can then dynamically request the required resources from the underlying batch system. In this paper we describe the architecture of the model and the experience of running many millions of ATLAS jobs on it.