Amazon SWF FAQs

To sign up for Amazon SWF, go to the Amazon SWF detail page and click the “Sign Up Now” button. If you do not have an Amazon Web Service account, you will be prompted to create one. After signing up, you can run a sample walkthrough in the AWS Management Console which takes you through the steps of running a simple image conversion application with Amazon SWF. You can also download the AWS Flow Framework samples to learn about the various features of the service. To start using Amazon SWF in your applications, please refer to the Amazon SWF documentation.

Yes. When you get started with Amazon SWF, you can try the sample walkthrough in the AWS Management Console which takes you through registering a domain and types, deploying workers and deciders and starting workflow executions. You can download the code for the workers and deciders used in this walkthrough, run them on your infrastructure and even modify them to build your own applications. You can also download the AWS Flow Framework samples, which illustrate the use of Amazon SWF for various use cases such as distributed data processing, Cron jobs and application stack deployment. By looking at the included source code, you can learn more about the features of Amazon SWF and how to use the AWS Flow Framework to build your distributed applications.

You can access SWF in any of the following ways:

• AWS SDK for Java, Ruby, .NET, and PHP
• AWS Flow Framework for Java (Included in the AWS SDK for Java)
• Amazon SWF web service APIs
• AWS Management Console

Registration is a one-time step that you perform for each different types of workflows and activities. You can register either programmatically or through the Amazon SWF Management Console. During registration, you provide unique type-ids for each activity and workflow type. You also provide default information that is used while running a workflow, such as timeout values and task distribution parameters.

In SWF, you define logical containers called domains for your application resources. Domains can only be created at the level of your AWS account and may not be nested. A domain can have any user-defined name. Each application resource, such as a workflow type, an activity type, or an execution, belongs to exactly one domain. During registration, you specify the domain under which a workflow or activity type should be registered. When you start an execution, it is automatically created in the same domain as its workflow type. The uniqueness of resource identifiers (e.g. type-ids, execution ID) is scoped to a domain, i.e. you may reuse identifiers across different domains.

You can use domains to organize your application resources so that they are easier to manage and do not inadvertently affect each other. For example, you can create different domains for your development, test, and production environments, and create the appropriate resources in each of them. Although you may register the same workflow type in each of these domains, it will be treated as a separate resource in each domain. You can change its settings in the development domain or administer executions in the test domain, without affecting the corresponding resources in the production domain.

The decider can be viewed as a special type of worker. Like workers, it can be written in any language and asks Amazon SWF for tasks. However, it handles special tasks called decision tasks. Amazon SWF issues decision tasks whenever a workflow execution has transitions such as an activity task completing or timing out. A decision task contains information on the inputs, outputs, and current state of previously initiated activity tasks. Your decider uses this data to decide the next steps, including any new activity tasks, and returns those to Amazon SWF. Amazon SWF in turn enacts these decisions, initiating new activity tasks where appropriate and monitoring them. By responding to decision tasks in an ongoing manner, the decider controls the order, timing, and concurrency of activity tasks and consequently the execution of processing steps in the application. SWF issues the first decision task when an execution starts. From there on, Amazon SWF enacts the decisions made by your decider to drive your execution. The execution continues until your decider makes a decision to complete it.

To help the decider in making decisions, SWF maintains an ongoing record on the details of all tasks in an execution. This record is called the history and is unique to each execution. A new history is initiated when an execution begins. At that time, the history contains initial information such as the execution’s input data. Later, as workers process activity tasks, Amazon SWF updates the history with their input and output data, and their latest state. When a decider gets a decision task, it can inspect the execution’s history. Amazon SWF ensures that the history accurately reflects the execution state at the time the decision task is issued. Thus, the decider can use the history to determine what has occurred in the execution and decide the appropriate next steps.

You use task lists to determine how tasks are assigned. Task lists are Amazon SWF resources into which initiated tasks are added and from which tasks are requested. Task lists are identified by user-defined names. A task list may have tasks of different type-ids, but they must all be either activity tasks or decision tasks. During registration, you specify a default task list for each activity and workflow type. Amazon SWF also lets you create task lists at run time. You create a task list simply by naming it and starting to use it. You use task lists as follows:

• While initiating an activity task, a decider can add it into a specific task list or request Amazon SWF to add it into the default task list for its activity type.
• While starting an execution, you can request Amazon SWF to add all of its decision tasks to a specific task list or to the default task list for the workflow type.
• While requesting tasks, deciders and workers specify which task list they want to receive tasks from. If a task is available in the list, SWF sends it in the response and also includes its type-id.

Based on the above, you control which task list a task gets added into and who asks for tasks from each list. Thus, you can ensure that workers and deciders only get the tasks that they understand.

AWS Flow Framework is a programming framework that enables you to develop Amazon SWF-based applications quickly and easily. It abstracts the details of task-level coordination and asynchronous interaction with simple programming constructs. Coordinating workflows in Amazon SWF involves initiating remote actions that take variable times to complete (e.g. activity tasks) and implementing the dependencies between them correctly.

AWS Flow Framework makes it convenient to express both facets of coordination through familiar programming concepts. For example, initiating an activity task is as simple as making a call to a method. AWS Flow Framework automatically translates the call into a decision to initiate the activity task and lets Amazon SWF assign the task to a worker, monitor it, and report back on its completion. The framework makes the outcome of the task, including its output data, available to you in the code as the return values from the method call. To express the dependency on a task, you simply use the return values in your code, as you would for typical method calls. The framework’s runtime will automatically wait for the task to complete and continue your execution only when the results are available. Behind the scenes, the framework’s runtime receives worker and decision tasks from Amazon SWF, invokes the relevant methods in your program at the right times, and formulates decisions to send back to Amazon SWF. By offering access to Amazon SWF through an intuitive programming framework, the AWS Flow Framework makes it possible to easily incorporate asynchronous and event driven programming in the development of your applications.

Typically poll based protocols require developers to find an optimal polling frequency. If developers poll too often, it is possible that many of the polls will be returned with empty results. This leads to a situation where much of the application and network resources are spent on polling without any meaningful outcome to drive the execution forward. If developers don’t poll often enough, then messages may be held for longer increasing application latencies.

To overcome the inefficiencies inherent in polling, Amazon SWF provides long-polling. Long-polling significantly reduces the number of polls that return without any tasks. When workers and deciders poll Amazon SWF for tasks, the connection is retained for a minute if no task is available. If a task does become available during that period, it is returned in response to the long-poll request. By retaining the connection for a period of time, additional polls that would also return empty during that period are avoided. With long-polling, your applications benefit with the security and flow control advantages of polling without sacrificing the latency and efficiency benefits offered by push-based web services.

Workers use standard HTTP GET requests to get tasks from Amazon SWF and to return the results. To use an existing web service as a worker, you can write a wrapper that gets tasks from Amazon SWF, invokes your web service’s APIs as appropriate, and returns the results back to Amazon SWF. In the wrapper, you translate input data provided in a task into the parameters for your web service’s API. Similarly, you also translate the output data from the web service APIs into results for the task and return those to Amazon SWF.

No, you can use any programming language to write a worker or a decider, as long as you can communicate with Amazon SWF using web service APIs. The AWS SDK is currently available in Java, .NET, PHP and Ruby. The AWS SDK for Java includes the AWS Flow Framework.

When you start new workflow executions you provide an ID for that workflow execution. This enables you to associate an execution with a business entity or action (e.g. customer ID, filename, serial number). Amazon SWF ensures that an execution’s ID is unique while it runs. During this time, an attempt to start another execution with the same ID will fail. This makes it convenient for you to satisfy business needs where no more than one execution can be running for a given business action, such as a transaction, submission or assignment. Consider a workflow that registers a new user on a website. When a user clicks the submit button, the user’s unique email address can be used to name the execution. If the execution already exists, the call to start the execution will fail. No additional code is needed to prevent conflicts as a result of the user clicking the button more than one when the registration is in progress.

Once the workflow execution is complete (either successfully or not), you can start another workflow execution with the same ID. This causes a new run of the workflow execution with the same execution ID but a different run ID. The run ID is generated by Amazon SWF and multiple executions that have the same workflow execution ID can be differentiated by the run ID. By allowing you to reuse workflow execution IDs in such a manner, Amazon SWF allows you to address use cases such as retries. For example, in the above user registration example, assume that the workflow execution failed when creating a database record for the user. You can start the workflow execution again with the same execution ID (user’s email address) and do not have to create a new ID for retrying the registration.

Amazon SWF lets you scale your applications by giving you full control over the number of workers that you run for each activity type and the number of instances that you run for a decider. By increasing the number of workers or decider instances, you increase the compute resources allocated for the corresponding processing steps and, thereby, the throughput for those steps. To auto-scale, you can use run-time data that Amazon SWF provides through its APIs. For example, Amazon SWF provides the number of tasks in a task list. Since an increase in this number implies that the workers are not keeping up with the load, you can spin up new workers automatically whenever the backlog of tasks crosses a threshold.

In addition to a Management Console, Amazon SWF provides a comprehensive set of visibility APIs. You can use these to get run-time information to monitor all your executions and to auto-scale your executions depending on load. You can get detailed data on each workflow type, such as the count of open and closed executions in a specified time range. Using the visibility APIs, you can also build your own custom monitoring applications.

Amazon SWF lets you search for executions through its Management Console and visibility APIs. You can search by various criteria, including the time intervals during which executions started or completed, current state (i.e. open or closed), and standard failure modes (e.g. timed out, terminated). To group workflow executions together, you can use upto 5 tags to associate custom text with workflow executions when you start them. In the AWS Management Console, you can use tags when searching workflow executions.

To find executions that may be stalled, you can start with a time-based search to hone in on executions that are running longer than expected. Next, you can inspect them to see task level details and determine if certain tasks have been running too long or have failed, or whether the decider has simply not initiated tasks. This can help you pinpoint the problem at a task-level.

Yes. Multiple applications can share a given activity type provided the applications and the activity are all registered within the same domain. To implement this, you can have different deciders initiate tasks for the activity type and add it to the task list that the workers for that activity poll on. The workers of that activity type will then get activity tasks from all the different applications. If you want to tell which application an activity task came from or if you want to deploy different sets of workers for different applications, you can use multiple task lists. Refer to How do I ensure that a worker or decider only gets tasks that it understands?

Yes. You can grant IAM users permission to access Amazon SWF. IAM users can only access the SWF domains and APIs that you specify.

Yes. Workers use standard HTTP GET requests to ask Amazon SWF for tasks and to return the computed results. Since workers always initiate requests to Amazon SWF, you do not have to configure your firewall to allow inbound requests.

Workers use standard HTTP GET requests to ask Amazon SWF for tasks and to return the computed results. Thus, you do not have to expose any endpoint for your workers. Furthermore, Amazon SWF only gives tasks to workers when the decider initiates those tasks. Since you write the decider, you have full control over when and how tasks are initiated, including the input data that gets sent with them to the workers.

Amazon SWF provides useful guarantees around task assignment. It ensures that a task is never duplicated and is assigned only once. Thus, even though you may have multiple workers for a particular activity type (or a number of instances of a decider), Amazon SWF will give a specific task to only one worker (or one decider instance). Additionally, Amazon SWF keeps at most one decision task outstanding at a time for a workflow execution. Thus, you can run multiple decider instances without worrying about two instances operating on the same execution simultaneously. These facilities enable you to coordinate your workflow without worrying about duplicate, lost, or conflicting tasks.

You can have a maximum of 10,000 workflow and activity types (in total) that are either registered or deprecated in each domain. You can have a maximum of 100 Amazon SWF domains (including registered and deprecated domains) in your AWS account.

At any given time, you can have a maximum of 100,000 open executions in a domain. There is no other limit on the cumulative number of executions that you run or on the number of executions retained by Amazon SWF.

Each workflow execution can run for a maximum of 1 year. Each workflow execution history can grow up to 25,000 events. If your use case requires you to go beyond these limits, you can use features Amazon SWF provides to continue executions and structure your applications using child workflow executions.

Amazon SWF does not take any special action if a workflow execution is idle for an extended period of time. Idle executions are subject to the timeouts that you configure. For example, if you have set the maximum duration for an execution to be 1 day, then an idle execution will be timed out if it exceeds the 1 day limit. Idle executions are also subject to the Amazon SWF limit on how long an execution can run (1 year).

Amazon SWF does not impose a specific limit on how long a worker can take to process a task. It enforces the timeout that you specify for the maximum duration for the activity task. Note that since Amazon SWF limits an execution to run for a maximum of 1 year, a worker cannot take longer than that to process a task.

Amazon SWF does not impose a specific limit on how long a task is kept before a worker polls for it. However, when registering the activity type, you can set a default timeout for how long Amazon SWF will hold on to activity tasks of that type. You can also specify this timeout or override the default timeout through your decider code when you schedule an activity task. Since Amazon SWF limits the time that a workflow execution can run to a maximum of 1 year, if a timeout is not specified, the task will not be kept longer than 1 year.

Yes, you can schedule up to 100 activity tasks in one decision and also issue several decisions one after the other.

There is no limit on the total number of activity tasks, signals, and timers used during a workflow execution. However at this time, you can only have a maximum of 1,000 open activity tasks per workflow execution. This includes activity tasks that have been initiated and activity tasks that are being processed by workers. Similarly there can be up to 1,000 open timers per workflow execution and up to 1,000 open child executions per workflow execution.

There is no limit on the total amount of data that is transferred during a workflow execution. However, Amazon SWF APIs impose specific maximum limits on parameters that are used to pass data within an execution. For example, the input data that is passed into a activity task and the input data that is sent with a signal can each be a maximum of 32,000 characters.

Amazon SWF retains the history of a completed execution for any number of days that you specify, up to a maximum of 90 days (i.e. approximately 3 months). During retention, you can access the history and search for the execution programmatically or through the console.

Beyond infrequent spikes, you may be throttled if you make a very large number of API calls in a very short period of time.

For service region availability, see the AWS Global Infrastructure Region Table.

Yes, Amazon SWF manages your workflow execution history and other details of your workflows across 3 availability zones so that your applications can continue to rely on Amazon SWF even if there are failures in one availability zone.

Please visit the AWS General Reference documentation for more information on access endpoints.

Except as otherwise noted, our prices are exclusive of applicable taxes and duties, including VAT and applicable sales tax. For customers with a Japanese billing address, use of AWS services is subject to Japanese Consumption Tax.

Learn more