FastReplica: Efficient Large File Distribution within Content Delivery Networks

Cherkasova, Ludmila ; Lee, Jangwon

HP Development Company

关键词: CDNs;    large-scale distributed network of servers;    content distribution;    scaleable and reliable algorithm;    wide-area testbed;    simulation;   

RP-ID  :  HPL-2003-43

学科分类：计算机科学（综合）

美国|英语

来源: HP Labs

【 摘 要 】

In this work, we consider a large-scale distributed network of servers and a problem of content distribution across it. We propose a novel algorithm, called FastReplica, for an efficient and reliable replication of large files in the Internet environment. There are a few basic ideas exploited in FastReplica. In order to replicate a large file among n nodes (n is in the range of 10-30 nodes), the original file is partitioned into n subfiles of equal size and each subfile is transferred to a different node in the group. After that, each node propagates its subfile to the remaining nodes in the group. Thus instead of the typical replication of an entire file to n nodes by using n Internet paths, connecting the original node to the replication group, FastReplica exploits n x n Internet paths within the replication group where each path is used for transferring 1/n-th of the file. We design a scalable and reliable FastReplica algorithm which can be used for replication of large files to a large group of nodes. The new method is simple and inexpensive. It does not require any changes or modifications to the existing Internet infrastructure, and at the same time, it significantly reduces the file replication time as we demonstrate through experiments on a prototype implementation of FastReplica in a wide-area testbed. Notes: To be published in the 4th Usenix Symposium on Internet Technologies and Systems (USITS 2003) 26-28 March 2003, Seattle, Washington 14 Pages