Recent Developments in the CernVM-File System Server Backend

16th International workshop on Advanced Computing and Analysis Techniques in physics research

物理学;计算机科学

Meusel, R.^1 ; Blomer, J.^1 ; Buncic, P.^1 ; Ganis, G.^1 ; Heikkila, S.^1

CERN PH-SFT, CERN, Geneva 23

CH-1211, Switzerland^1

关键词: Computing community;    De facto standard;    Direct manipulation;    Feed-back loop;    File system snapshots;    Garbage collection;    HEP experiments;    Repository data;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/608/1/012031/pdf
DOI  :  10.1088/1742-6596/608/1/012031

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

The CernVM File System (CernVM-FS) is a snapshotting read-only file system designed to deliver software to grid worker nodes over HTTP in a fast, scalable and reliable way. In recent years it became the de-facto standard method to distribute HEP experiment software in the WLCG and starts to be adopted by other grid computing communities outside HEP. This paper focusses on the recent developments of the CernVM-FS Server, the central publishing point of new file system snapshots. Using a union file system, the CernVM-FS Server allows for direct manipulation of a (normally read-only) CernVM-FS volume with copy-on-write semantics. Eventually the collected changeset is transformed into a new CernVM-FS snapshot, constituting a transactional feedback loop. The generated repository data is pushed into a content addressable storage requiring only a RESTful interface and gets distributed through a hierarchy of caches to individual grid worker nodes. Additonally we describe recent features, such as file chunking, repository garbage collection and file system history that enable CernVM- FS for a wider range of use cases.