A Survey on Distributed File System Technology

16th International workshop on Advanced Computing and Analysis Techniques in physics research

物理学;计算机科学

Blomer, J.^1

CERN, Genève 23

CH-1211, Switzerland^1

关键词: Distributed consensus;    Distributed coordination;    Distributed file systems;    Distributed Hash Table;    Distributed process;    Distributed storage;    Resilience to failure;    Storage architectures;   

Others  :  https://iopscience.iop.org/article/10.1088/1742-6596/608/1/012039/pdf
DOI  :  10.1088/1742-6596/608/1/012039

学科分类：计算机科学（综合）

来源: IOP

【 摘 要 】

Distributed file systems provide a fundamental abstraction to location-transparent, permanent storage. They allow distributed processes to co-operate on hierarchically organized data beyond the life-time of each individual process. The great power of the file system interface lies in the fact that applications do not need to be modified in order to use distributed storage. On the other hand, the general and simple file system interface makes it notoriously difficult for a distributed file system to perform well under a variety of different workloads. This has lead to today's landscape with a number of popular distributed file systems, each tailored to a specific use case. Early distributed file systems merely execute file system calls on a remote server, which limits scalability and resilience to failures. Such limitations have been greatly reduced by modern techniques such as distributed hash tables, content-addressable storage, distributed consensus algorithms, or erasure codes. In the light of upcoming scientific data volumes at the exabyte scale, two trends are emerging. First, the previously monolithic design of distributed file systems is decomposed into services that independently provide a hierarchical namespace, data access, and distributed coordination. Secondly, the segregation of storage and computing resources yields to a storage architecture in which every compute node also participates in providing persistent storage.