Prototype of a production system for Cherenkov Telescope Array with DIRAC

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Arrabito, L.^1 ; Bregeon, J.^1 ; Haupt, A.^2,3 ; Graciani Diaz, R.^4 ; Stagni, F.^5 ; Tsaregorodtsev, A.^6

Laboratoire Univers et Particules, Université de Montpellier II Place Eugène Bataillon, CNRS/IN2P3, CC 72, Montpellier

F-34095, France^1

Deutsches Elektronen-Synchrotron, Platanenallee 6, Zeuthen

15738, Germany^2

Department of Physics and Electrical Engineering, Linnaeus University, Växjö

35195, Sweden^3

University of Barcelona, Diagonal 647, Barcelona

ES-08028, Spain^4

CERN, Switzerland^5

Centre de Physique des Particules de Marseille, CNRS/IN2P3, 163 Av de Luminy Case 902, Marseille

13288, France^6

关键词: Cherenkov telescope arrays;    Data management system;    Distributed environments;    Distributed heterogeneous computing;    Distributed infrastructure;    Imaging atmospheric Cherenkov telescopes;    Transformation systems;    Very-high-energy gamma-ray astronomies;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/3/032001/pdf
DOI  :  10.1088/1742-6596/664/3/032001

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The Cherenkov Telescope Array (CTA) - an array of many tens of Imaging Atmospheric Cherenkov Telescopes deployed on an unprecedented scale - is the next generation instrument in the field of very high energy gamma-ray astronomy. CTA will operate as an open observatory providing data products to the scientific community. An average data stream of about 10 GB/s for about 1000 hours of observation per year, thus producing several PB/year, is expected. Large CPU time is required for data-processing as well for massive Monte Carlo simulations needed for detector calibration purposes. The current CTA computing model is based on a distributed infrastructure for the archive and the data off-line processing. In order to manage the off-line data-processing in a distributed environment, CTA has evaluated the DIRAC (Distributed Infrastructure with Remote Agent Control) system, which is a general framework for the management of tasks over distributed heterogeneous computing environments. In particular, a production system prototype has been developed, based on the two main DIRAC components, i.e. the Workload Management and Data Management Systems. After three years of successful exploitation of this prototype, for simulations and analysis, we proved that DIRAC provides suitable functionalities needed for the CTA data processing. Based on these results, the CTA development plan aims to achieve an operational production system, based on the DIRAC Workload Management System, to be ready for the start of CTA operation phase in 2017-2018. One more important challenge consists of the development of a fully automatized execution of the CTA workflows. For this purpose, we have identified a third DIRAC component, the so-called Transformation System, which offers very interesting functionalities to achieve this automatisation. The Transformation System is a 'data-driven' system, allowing to automatically trigger data-processing and data management operations according to pre-defined scenarios. In this paper, we present a brief summary of the DIRAC evaluation done so far, as well as the future developments planned for the CTA production system. In particular, we will focus on the developments of CTA automatic workflows, based on the Transformation System. As a result, we also propose some design optimizations of the Transformation System, in order to fully support the most complex workflows, envisaged in the CTA processing.