RunJRun: Parallel Processing for Java on EC2 Instances
RunJRun
is a very simple system for doing parallel processing in Java, using Amazon Elastic Compute
Cloud (EC2) instances as compute nodes.
The basic compute unit is a Runnable, Serializable Java object, a 'task' for short. A user submits a list of such tasks to RunJRun. Each task then has its run() method invoked on an EC2 instance. Optionally, the 'cooked' task object can be stored in S3.
RunJRun uses SQS and S3 to make tasks available to compute nodes. These services scale very well, so RunJRun can handle large numbers of tasks and EC2 instances.
A RunJRun compute node executes one run() method at a time, so that a task has exclusive use of an EC2 instance. (To fully utilize EC2 instances, you may want to make your tasks multi-threaded. See Multi-thread tasks in the RunJRun manual).
RunJRun is written in Java and is implemented as an api.
RunJRun itself is free, open-source software. To use it, you'll need an Amazon Machine Image (AMI) that has the RunJRun compute node software installed. Several such AMIs are available. These are DevPay AMIs - you'll be charged $0.03/hour to use them, in addition to Amazon's regular charges. For details, please go to RunJRun AMIs. Or you can construct your own RunJRun AMI.
For succinct instructions on using RunJRun, see Quick Start. For more details, see the manual.
RunJRun links:
The basic compute unit is a Runnable, Serializable Java object, a 'task' for short. A user submits a list of such tasks to RunJRun. Each task then has its run() method invoked on an EC2 instance. Optionally, the 'cooked' task object can be stored in S3.
RunJRun uses SQS and S3 to make tasks available to compute nodes. These services scale very well, so RunJRun can handle large numbers of tasks and EC2 instances.
A RunJRun compute node executes one run() method at a time, so that a task has exclusive use of an EC2 instance. (To fully utilize EC2 instances, you may want to make your tasks multi-threaded. See Multi-thread tasks in the RunJRun manual).
RunJRun is written in Java and is implemented as an api.
RunJRun itself is free, open-source software. To use it, you'll need an Amazon Machine Image (AMI) that has the RunJRun compute node software installed. Several such AMIs are available. These are DevPay AMIs - you'll be charged $0.03/hour to use them, in addition to Amazon's regular charges. For details, please go to RunJRun AMIs. Or you can construct your own RunJRun AMI.
For succinct instructions on using RunJRun, see Quick Start. For more details, see the manual.
RunJRun links:
