20th International Conference on Computing in High Energy and Nuclear Physics | |
XRootd, disk-based, caching proxy for optimization of data access, data placement and data replication | |
物理学;计算机科学 | |
Bauerdick, L.A.T.^1 ; Bloom, K.^3 ; Bockelman, B.^3 ; Bradley, D.C.^4 ; Dasu, S.^4 ; Dost, J.M.^2 ; Sfiligoi, I.^2 ; Tadel, A.^2 ; Tadel, M.^2 ; Wuerthwein, F.^2 ; Yagil, A.^2 | |
Fermilab, Batavia | |
IL | |
60510-5011, United States^1 | |
UC San Diego, San Diego | |
CA | |
92093, United States^2 | |
University of Nebraska-Lincoln, Lincoln | |
NE | |
68588, United States^3 | |
University of Wisconsin-Madison, Madison | |
WI | |
53706, United States^4 | |
关键词: Data federation; Data placement; Data replication; File access; Hadoop distributed file systems; Implementation support; Network access; Pre-production; | |
Others : https://iopscience.iop.org/article/10.1088/1742-6596/513/4/042044/pdf DOI : 10.1088/1742-6596/513/4/042044 |
|
学科分类:计算机科学(综合) | |
来源: IOP | |
【 摘 要 】
Following the success of the XRootd-based US CMS data federation, the AAA project investigated extensions of the federation architecture by developing two sample implementations of an XRootd, disk-based, caching proxy. The first one simply starts fetching a whole file as soon as a file open request is received and is suitable when completely random file access is expected or it is already known that a whole file be read. The second implementation supports on-demand downloading of partial files. Extensions to the Hadoop Distributed File System have been developed to allow for an immediate fallback to network access when local HDFS storage fails to provide the requested block. Both cache implementations are in pre-production testing at UCSD.
【 预 览 】
Files | Size | Format | View |
---|---|---|---|
XRootd, disk-based, caching proxy for optimization of data access, data placement and data replication | 974KB | download |