JobCenter: an open source, cross-platform, and distributed job queue management system optimized for scalability and versatility

Source Code for Biology and Medicine

Michael Riffle²  Daniel Jaschob¹ 

[1] Department of Biochemistry, University of Washington, 1705 NE Pacific St, Box 357350, Seattle, WA, 98195, USA;Department of Genome Sciences, University of Washington, Seattle, WA, 98195, USA

关键词: Batch processing;    Job scheduler;    Grid computing;    Cross-platform;    Open-source;    Distributed;    Job management;    Computational biology;    Bioinformatics;    Computing;   

Others  :  806236
DOI  :  10.1186/1751-0473-7-8

 received in 2011-10-07, accepted in 2012-07-25,  发布年份 2012

【 摘 要 】

Background

Laboratories engaged in computational biology or bioinformatics frequently need to run lengthy, multistep, and user-driven computational jobs. Each job can tie up a computer for a few minutes to several days, and many laboratories lack the expertise or resources to build and maintain a dedicated computer cluster.

Results

JobCenter is a client–server application and framework for job management and distributed job execution. The client and server components are both written in Java and are cross-platform and relatively easy to install. All communication with the server is client-driven, which allows worker nodes to run anywhere (even behind external firewalls or “in the cloud”) and provides inherent load balancing. Adding a worker node to the worker pool is as simple as dropping the JobCenter client files onto any computer and performing basic configuration, which provides tremendous ease-of-use, flexibility, and limitless horizontal scalability. Each worker installation may be independently configured, including the types of jobs it is able to run. Executed jobs may be written in any language and may include multistep workflows.

Conclusions

JobCenter is a versatile and scalable distributed job management system that allows laboratories to very efficiently distribute all computational work among available resources. JobCenter is freely available athttp://code.google.com/p/jobcenter/ webcite.

【 授权许可】

   
2012 Jaschob and Riffle; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

【 参考文献 】

• [1]Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ: Basic local alignment search tool. J Mol Biol 1990, 215(3):403-410.
• [2]Perkins DN, Pappin DJ, Creasy DM, Cottrell JS: Probability-based protein identification by searching sequence databases using mass spectrometry data. Electrophoresis 1999, 20(18):3551-3567.
• [3]Yates JR 3rd, Eng JK, Mc Cormack AL: Mining genomes: correlating tandem mass spectra of modified and unmodified peptides to sequences in nucleotide databases. Anal Chem 1995, 67(18):3202-3210.
• [4]Deutsch EW, Mendoza L, Shteynberg D, Farrah T, Lam H, Tasman N, Sun Z, Nilsson E, Pratt B, Prazen B, et al.: A guided tour of the Trans-Proteomic Pipeline. Proteomics 2010, 10(6):1150-1159.
• [5]Reynolds SM, Kall L, Riffle ME, Bilmes JA, Noble WS: Transmembrane topology and signal peptide prediction using dynamic bayesian networks. PLoS Comput Biol 2008, 4(11):e1000213.
• [6]Lipman DJ, Pearson WR: Rapid and sensitive protein similarity searches. Science 1985, 227(4693):1435-1441.
• [7]Riffle M, Davis TN: The Yeast Resource Center Public Image Repository: A large database of fluorescence microscopy images. BMC Bioinformatics 2010, 11:263. BioMed Central Full Text
• [8]Riffle M, Malmstrom L, Davis TN: The Yeast Resource Center Public Data Repository. Nucleic Acids Res 2005, 33(Database issue):D378-D382.