NASA Archives and Data Stewardship in the Cloud with Cumulus

Frederick, Lauren

关键词: EOS DATA AND INFORMATION SYSTEM;    DATA STORAGE;    BIG DATA;    DATA MANAGEMENT;    EARTH OBSERVING SYSTEM (EOS);    OPEN SOURCE LICENSING (COMPUTERS);    COMPUTER PROGRAMMING;    NASA PROGRAMS;   

RP-ID  :  GSFC-E-DAA-TN64408

美国|英语

来源: NASA Technical Reports Server

【 摘 要 】

NASA's Earth Observing System Data and Information System (EOSDIS) houses nearly 30PB (petabytes) of critical Earth Science data and with upcoming missions is expected to balloon to between 200PBs-300PBs over the next seven years. The magnitude of data collected makes it infeasible to download data and process it locally, forcing us to re-think how we store and work with earth science data. NASA has looked to the cloud to address this, building its open source Cumulus software to manage the ingest of diverse data in a wide variety of formats into the cloud and provide services to manage and access the data. In this talk, we will describe how Cumulus provides common features needed to manage a cloud archive in the realm of ingest, data stewardship, and cost-controlled distribution of science data to users and services.