AWS News Blog
New – Amazon Simple Workflow Recipes
|
We launched Amazon Simple Workflow (SWF) earlier this year, introducing a service designed to help developers automate the coordination of work in applications for better scalability and performance. Coordination of work in an application becomes particularly onerous for developers to build when the application needs to manage high volume and/or multiple streams of work concurrently across a collection of servers. For example, developers who build applications that deal at scale with image conversion, encoding, or data analysis often spend significant time writing code that coordinates how the work gets executed. Without SWF, a developer would need to write custom code that manages the execution state, concurrency, and distribution of work. Aside from more code to develop and maintain, this kind of work coordination is complex to write. A developer would need to build code to manage the state of process execution, define how the application reacts to failed or delayed processes, and manage work streams that can complete at different times.
SWF automates the coordination of work for applications, reducing the amount of code and complexity that developers write to handle work that executes across multiple processes and servers. SWF provides a programming model to simplify how developers express work coordination business logic in the application, as well as a service to manage the coordination of work at runtime. Customers today are taking advantage of SWF to coordinate high-volume, low-latency processing that spans data centers. For example, the developers and architects at NASA JPL used SWF to automate a complex image-processing pipeline for the Mars Science Lander (Curiosity). The case study contains a lot of helpful information about how (and why) NASA used SWF. Other customers use SWF for applications that range from automating scheduled jobs run by several EC2 instances to video encoding to stringing together the steps required to get packages from a warehouse to a delivery truck.
To adopt SWF, customers need to add code to their application that defines work coordination logic in a decider (specifying work sequencing, timing, and failure conditions) and which application components are executing work in activities that live within workflows. Once these changes are in place, the SWF service will coordinate the work automatically on behalf of the application. Developers can use any programming language when programming directly against the SWF service API to define deciders, activities, and workflows. We also provide a Java client AWS Flow Framework, which simplifies common programming tasks like retry logic for failed or delayed work.
Flow Framework Recipes
To make it easier for customers to use SWF, we are introducing 17 AWS Flow Framework recipes that show best practices for adding SWF work coordination logic to your application. These recipes are Java code samples with JUnit tests that use the AWS Flow Framework to perform common programming steps.
The recipes start by showing you how to add common workflow coordination logic, like repeatedly executing a unit of work (an activity) in your application iteratively or recursively until a condition like time or number of retries is met. Next, the recipes give guidance on setting up more involved coordination logic, like executing multiple processes of the same work in your application in parallel or starting multiple processes of the same work and taking the results of the process that finishes first. This type of parallel processing helps developers scale high-volume processing jobs by leveraging concurrent process execution while still exerting control over the work. We also show you how to put work on hold until an action occurs elsewhere in the application. Finally, the recipes show how you can define retry logic for failed or delayed work in an application.
Getting Started
The AWS Flow Framework recipes take the form of 11 Java packages, some of which contain multiple recipes which are commonly used together. The recipes build upon the AWS Flow Framework and take advantage of a number of its data types and features including Promise<T>, Settable<T>, and Asynchronous methods.
If you’d like a little background on the programming concepts in Simple Workflow before you start, you can check out the Amazon Simple Workflow Service Developer Guide’s Introduction, Getting Set Up, and Basic Concepts sections. You can also check out my blog post introducing Simple Workflow.
We have plenty of documentation to help you get started, along with instructions to run the recipes using Eclipse and Ant. You can find everything you need in the AWS Flow Framework recipes package. If you need any help, feel free to post a question to the Amazon Simple Workflow developer forum.
I hope that you find the recipes useful. Let me know what you think!
— Jeff;