A Framework for Analyzing and Improving Content-Based Chunking Algorithms

Eshghi, Kave ; Tang, Hsiu Khuern

HP Development Company

关键词: stateless chunking algorithm;    hash function;    storage overhead;    archival file systems;    low bandwidth networks;    file system;   

RP-ID  :  HPL-2005-30R1

学科分类：计算机科学（综合）

美国|英语

来源: HP Labs

【 摘 要 】

We present a framework for analyzing content-based chunking algorithms, as used for example in the Low Bandwidth Networked File System. We use this framework for the evaluation of the basic sliding window algorithm, and its two known variants. We develop a new chunking algorithm that performs significantly better than the known algorithms on real, non-random data. Notes: 10 Pages