Dynamic provisioning of local and remote compute resources with OpenStack

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Giffels, M.^1 ; Hauth, T.^1 ; Polgart, F.^1 ; Quast, G.^1

Karlsruhe Institute of Technology, Institut für Experimentelle Kernphysik, Wolfgang-Gaede-Str. 1, Karlsruhe

76131, Germany^1

关键词: Computing capability;    Computing resource;    Dynamic provisioning;    High energy physics experiments;    Operational experience;    Software environments;    Virtual machine management;    Virtualization technologies;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/2/022022/pdf
DOI  :  10.1088/1742-6596/664/2/022022

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

Modern high-energy physics experiments rely on the extensive usage of computing resources, both for the reconstruction of measured events as well as for Monte-Carlo simulation. The Institut fur Experimentelle Kernphysik (EKP) at KIT is participating in both the CMS and Belle experiments with computing and storage resources. In the upcoming years, these requirements are expected to increase due to growing amount of recorded data and the rise in complexity of the simulated events. It is therefore essential to increase the available computing capabilities by tapping into all resource pools. At the EKP institute, powerful desktop machines are available to users. Due to the multi-core nature of modern CPUs, vast amounts of CPU time are not utilized by common desktop usage patterns. Other important providers of compute capabilities are classical HPC data centers at universities or national research centers. Due to the shared nature of these installations, the standardized software stack required by HEP applications cannot be installed. A viable way to overcome this constraint and offer a standardized software environment in a transparent manner is the usage of virtualization technologies. The OpenStack project has become a widely adopted solution to virtualize hardware and offer additional services like storage and virtual machine management. This contribution will report on the incorporation of the institute's desktop machines into a private OpenStack Cloud. The additional compute resources provisioned via the virtual machines have been used for Monte-Carlo simulation and data analysis. Furthermore, a concept to integrate shared, remote HPC centers into regular HEP job workflows will be presented. In this approach, local and remote resources are merged to form a uniform, virtual compute cluster with a single point-of-entry for the user. Evaluations of the performance and stability of this setup and operational experiences will be discussed.