New – AWS Step Functions – Build Distributed Applications Using Visual Workflows

We want to make it even easier for you to build complex, distributed applications by connecting multiple web and microservices. Whether you are implementing a complex business process or setting up a processing pipeline for photo uploads, we want you to focus on the code instead of on the coordination. We want you to be able to build reliable applications that are robust, scalable, and cost-effective, while you use the tools and libraries that you are already familiar with.

How does that sound?

Introducing AWS Step Functions
Today we are launching AWS Step Functions to allow you to do exactly what I described above. You can coordinate the components of your application as series of steps in a visual workflow. You create state machines in the Step Functions Console to specify and execute the steps of your application at scale.

Each state machine defines a set of states and the transitions between them. States can be activated sequentially or in parallel; Step Functions will make sure that all parallel states run to completion before moving forward. States perform work, make decisions, and control progress through the state machine.

Here’s a state machine that includes a little bit of everything:

Multiple copies of each state machine can be running independently at the same time; each copy is called an execution. Step Functions will let you run thousands of execution concurrently so you can scale to any desired level.

There are two different ways to specify what you want to happen when a state is run. First, you can supply a Lambda function that will be synchronously invoked when the state runs. Second, you can supply the name of an Activity. This is a reference to a long-running worker function that polls (via the API) for work to be done. Either way, the code is supplied with a JSON statement as input, and is expected to return another JSON statement as output.

As part of your state machine, you can specify error handling behavior and retry logic. This allows you to build robust multi-step apps that will run smoothly even if transient issues in one part of your code cause a momentary failure.

Quick Tour
Let’s set up a state machine through the AWS Management Console. Keep in mind that production applications will most likely use the AWS Step Functions API (described below) to create and run state machines.

I start by creating and saving a simple Lambda function:

While I am there I also capture the function’s ARN:

Then I go over to the AWS Step Functions Console and click on Create a State Machine. I enter a name (MyStateMachine), and I can click on one of the blueprints to get a running start:

I start with Hello World and use elements of Parallel to create this JSON model of my state machine (read the Amazon States Language spec to learn more):

{
  "Comment": "A simple example of the Steps language using an AWS Lambda Function",
  "StartAt": "Hello",

  "States": {
    "Hello": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:eu-west-1:99999999999:function:HelloWord_Step",
      "Next": "Parallel"
    },

    "Parallel": {
      "Type": "Parallel",
      "Next": "Goodbye",
      "Branches": [
        {
          "StartAt": "p1",
          "States": {
            "p1": {
                  "Type": "Task",
                  "Resource": "arn:aws:lambda:eu-west-1:9999999999:function:HelloWord_Step",
              "End": true
            }
          }
        },

        {
          "StartAt": "p2",
          "States": {
            "p2": {
                  "Type": "Task",
                  "Resource": "arn:aws:lambda:eu-west-1:99999999999:function:HelloWord_Step",
              "End": true
            }
          }
        }
      ]
    },

    "Goodbye": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:eu-west-1:99999999999:function:HelloWord_Step",
      "End": true
    }
  }
}

I click on Preview to see it graphically:

Then I select the IAM role that Step Functions thoughtfully created for me:

And I am all set! Now I can execute my state machine from the console; I can start it off with a block of JSON that is passed to the first function:

The state machine starts to execute as soon as I click on Start Execution. I can follow along and watch as execution flows from state to state:

I can visit the Lambda Console and see that my function ran four times as expected (I was pressed for time and didn’t bother to create four separate functions):

AWS Step Functions records complete information about each step and I can access it from the Step Console:

AWS Step Functions API
As I mentioned earlier, most of your interaction with AWS Step Functions will happen through the APIs. Here’s a quick overview of the principal functions:

• CreateStateMachine – Create a new state machine, given a JSON description.
• ListStateMachines – Get a list of state machines.
• StartExecution – Run (asynchronously) a state machine.
• DescribeExecution – Get information about an execution.
• GetActivityTask – Poll for new tasks to run (used by long-running workers).

You could arrange to run a Lambda function every time a new object is uploaded to an S3 bucket. This function can then kick off a state machine execution by calling StartExecution. The state machine could (as an example) validate the image, generate multiple sizes and formats in parallel, check for particular types of content, and update a database entry.

The same functionality is also available from the AWS Command Line Interface (AWS CLI).

Development Tools
You can use our new statelint gem to check your hand or machine-generated JSON for common errors including unreachable states and the omission of a terminal state.

Download it from the AWS Labs GitHub repo ( it will also be available on RubyGems) and install it like this:

$ sudo gem install j2119-0.1.0.gem statelint-0.1.0.gem

Here’s what happens if you have a problem:

$ statelint my_state.json
2 errors:
 State Machine.States.Goodbye does not have required field "Next"
 No terminal state found in machine at State Machine.States

And if things look good:

$ statelint my_state.json
$

Available Now
AWS Step Functions is available now and you can start using it today in the US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Ireland), and Asia Pacific (Tokyo) Regions.

As part of the AWS Free Tier, you can perform up to 4,000 state transitions per month at no charge. After that, you pay $0.025 for ever 1,000 state transitions.

You can learn more during our webinar on December 16th. Register here.

— Jeff;