High Speed Raster Image Streaming For Digital Presses Using the

Perry, Russell

HP Development Company

关键词: Print;    VDP;    Hadoop;    Binary Integer Programming;   

RP-ID  :  HPL-2009-345

学科分类：计算机科学（综合）

美国|英语

来源: HP Labs

【 摘 要 】

An application of the distributed Hadoop file system to very high rate variable data printing is described. The raster image processing of a large variable data document is represented as a MapReduce process. The key challenge addressed by this paper is how to stream the resulting raster images off the Hadoop file system to a digital press at multi-gigabit data rates. To achieve this, efficient scheduling of the order in which file blocks are read by the client is beneficial. An approach to scheduling based on binary integer programming is described which generates more efficient schedules compared to a na*ive approach. The scheduling model allows the exploration of system design choices and helps to identify file block distributions that are problematic to read at high rates. Measured stream rates approaching 4Gb/s were achieved which is close to the required rate for streaming pages containing rich designs to a digital press. This required only a minor extension to the Hadoop client to allow file blocks to be read in parallel from the Hadoop data nodes.