CloudBurst

CloudBurst provides highly-sensitive short read mapping with MapReduce.

Details

Submitted By: Jai@AWS
Created On: March 31, 2009 4:08 AM GMT
Last Updated: April 2, 2009 8:53 PM GMT

Provided by Michael Schatz, Center for Bioinformatics and Computational Biology, University of Maryland

Next-generation DNA sequencing machines are generating an enormous amount of sequence data, placing unprecedented demands on traditional single-processor read mapping algorithms. CloudBurst is a new parallel read-mapping algorithm optimized for mapping next-generation sequence data to the human genome and other reference genomes, for use in a variety of biological analyses including SNP discovery, genotyping, and personal genomics. It is modeled after the short read mapping program RMAP, and reports either all alignments or the unambiguous best alignment for each read with any number of mismatches or differences. This level of sensitivity could be prohibitively time consuming, but CloudBurst uses the open-source Hadoop implementation of MapReduce to parallelize execution using multiple compute nodes.

CloudBurst's running time scales linearly with the number of reads mapped, and with near linear speedup as the number of processors increases. In a 24-processor core configuration, CloudBurst is up to 30 times faster than RMAP executing on a single core, while computing an identical set of alignments. In a large remote compute clouds with 96 cores, CloudBurst reduces the running time from hours to mere minutes for typical jobs involving mapping of millions of short reads to the human genome. CloudBurst is available open-source as a model for parallelizing other bioinformatics algorithms with MapReduce.

Source Location on Amazon S3: elasticmapreduce/samples/cloudburst/code/cloudburst-bio.tar.gz
Source License: Apache License, Version 2.0
How to Run this Application: You can run this application using AWS Management Console or Command Line Tools
Default Input Parameters: refath: s3n://elasticmapreduce/samples/cloudburst/input/s_suis.br
(The file s_suis.br has the complete genome for a bacteria called Streptococcus suis)
qrypath: s3n://elasticmapreduce/samples/cloudburst/input/100k.br
(100k.br is a collection of 100,000 short DNA sequences called reads, each 36 characters long)
outpath: s3n://<yourbucket>/cloudburst/output/<date>
MIN_READ_LEN: 36
MAX_READ_LEN: 36
K: 3
SEED_LEN: 9
FLANK_LEN: 30
ALLOW_DIFFERENCES: 0
FILTER_ALIGNMENTS: true
NUM_MAP_TASKS: 240
NUM_REDUCE_TASKS: 48
BLOCK_SIZE: 128
REDUNDANCY: 16
Cloud Burst for Computer Scientists: http://apps.sourceforge.net/mediawiki/cloudburst-bio/index.php?title=CloudBurst_for_Computer_Scientists
Further reading: http://apps.sourceforge.net/mediawiki/cloudburst-bio/index.php?title=CloudBurst
©2014, Amazon Web Services, Inc. or its affiliates. All rights reserved.