Simplifying the development of portable, scalable, and reproducible workflows

eLife

Stephen R Piccolo¹  Zachary E Ence¹  Elizabeth C Anderson¹  Jeffrey T Chang²  Andrea H Bild³ 

[1] Department of Biology, Brigham Young University, Provo, United States;Department of Integrative Biology and Pharmacology, University of Texas Health Science Center at Houston, Houston, United States;Department of Medical Oncology and Therapeutics, City of Hope Comprehensive Cancer Institute, Monrovia, United States;

关键词: computational workflows;    research reproducibility;    learn by example;    Web application;    Common Workflow Language;    command-line software;    None;   

DOI  :  10.7554/eLife.71069

来源: eLife Sciences Publications, Ltd

【 摘 要 】

Command-line software plays a critical role in biology research. However, processes for installing and executing software differ widely. The Common Workflow Language (CWL) is a community standard that addresses this problem. Using CWL, tool developers can formally describe a tool’s inputs, outputs, and other execution details. CWL documents can include instructions for executing tools inside software containers. Accordingly, CWL tools are portable—they can be executed on diverse computers—including personal workstations, high-performance clusters, or the cloud. CWL also supports workflows, which describe dependencies among tools and using outputs from one tool as inputs to others. To date, CWL has been used primarily for batch processing of large datasets, especially in genomics. But it can also be used for analytical steps of a study. This article explains key concepts about CWL and software containers and provides examples for using CWL in biology research. CWL documents are text-based, so they can be created manually, without computer programming. However, ensuring that these documents conform to the CWL specification may prevent some users from adopting it. To address this gap, we created ToolJig, a Web application that enables researchers to create CWL documents interactively. ToolJig validates information provided by the user to ensure it is complete and valid. After creating a CWL tool or workflow, the user can create ‘input-object’ files, which store values for a particular invocation of a tool or workflow. In addition, ToolJig provides examples of how to execute the tool or workflow via a workflow engine. ToolJig and our examples are available at https://github.com/srp33/ToolJig.

【 授权许可】

CC BY