CMS distributed data analysis with CRAB3

21st International Conference on Computing in High Energy and Nuclear Physics

物理学;计算机科学

Mascheroni, M.^1 ; Balcas, J.^2 ; Belforte, S.^3 ; Bockelman, B.P.^4 ; Hernandez, J.M.^5 ; Ciangottini, D.^6 ; Konstantinov, P.B.^7 ; Silva, J.M.D.^8 ; Ali, M.A.B.M.^9 ; Melo, A.M.^10 ; Riahi, H.^11 ; Tanasijczuk, A.J.^12 ; Yusli, M.N.B.^9 ; Wolf, M.^13 ; Woodard, A.E.^13 ; Vaandering, E.^14

Universita e INFN, Sezione di Milano-Bicocca, Italy^1

Vilnius University, Italy^2

Universita e INFN, Sezione di Trieste, Italy^3

University of Nebraska-Lincoln, United States^4

CIEMAT, Madrid, Spain^5

Universita e INFN, Sezione di Perugia, Italy^6

Bulgarian Academy of Sciences, Bulgaria^7

UNESP - Universidade Estadual Paulista, Brazil^8

University of Malaya, Brazil

(10) Vanderbilt University, United States

(11) CERN, Switzerland

(12) University of California San Diego, United States

(13) University of Notre Dame, United States

(14) FNAL, United States^9

关键词: Commissioning phase;    Distributed data analysis;    Distributed workflow;    Grid infrastructures;    Resource provisioning;    Storage location;    Technical details;    Temporary storage;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/664/6/062038/pdf
DOI  :  10.1088/1742-6596/664/6/062038

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The CMS Remote Analysis Builder (CRAB) is a distributed workflow management tool which facilitates analysis tasks by isolating users from the technical details of the Grid infrastructure. Throughout LHC Run 1, CRAB has been successfully employed by an average of 350 distinct users each week executing about 200,000 jobs per day. CRAB has been significantly upgraded in order to face the new challenges posed by LHC Run 2. Components of the new system include 1) a lightweight client, 2) a central primary server which communicates with the clients through a REST interface, 3) secondary servers which manage user analysis tasks and submit jobs to the CMS resource provisioning system, and 4) a central service to asynchronously move user data from temporary storage in the execution site to the desired storage location. The new system improves the robustness, scalability and sustainability of the service. Here we provide an overview of the new system, operation, and user support, report on its current status, and identify lessons learned from the commissioning phase and production roll-out.