An On-Demand Processing Framework for Faster Remote Sensing Big Data Analysis

6th Annual 2018 International Conference on Geo-Spatial Knowledge and Intelligence

Huang, Zhenchun^1

Department of Computer Science and Technology, Tsinghua University, Beijing, China^1

关键词: Analysis models;    Data analysis models;    Fast analysis;    Layered architecture;    Prototype system;    Raw data files;    Remote sensing analysis;    Remote sensing data;   

Others  :  https://iopscience.iop.org/article/10.1088/1755-1315/234/1/012060/pdf
DOI  :  10.1088/1755-1315/234/1/012060

来源: IOP

【 摘 要 】

Remote sensing data are considered as "big geo data" because of their huge data volume, significant heterogeneity and challenge of fast analysis. The traditional remote sensing analysis workflows make earth scientists to download raw data files to local workstations before processing them for science discoveries. The data transfer often costs a lot of time and slows down the analysis workflows. Due to results of remote sensing data analysis models are usually much smaller than raw data to be processed in most cases, "on-demand processing", which tries to upload and run data analysis models near the remote sensing data, can make the remote sensing analysis workflows faster. In this paper, an on-demand remote sensing data processing framework is proposed based on a three-layered architecture for faster remote sensing data analysis workflows. The evaluation on a prototype system shows that the on-demand processing framework can accelerate the execution of analysis models significantly by reducing data transfers, especially for those analysis workflows which transfer data through low bandwidth Internet.