How to Handle Errors in Serverless Apps with AWS Step Functions and AWS Lambda

TUTORIAL

Overview

In this tutorial, you will learn how to use AWS Step Functions to handle workflow runtime errors. AWS Step Functions is a serverless orchestration service that lets you easily coordinate multiple Lambda functions into flexible workflows that are easy to debug and easy to change. AWS Lambda is a compute service that lets you run code without provisioning or managing servers.

Lambda functions can occasionally fail, such as when an unhandled exception is raised, when they run longer than the configured timeout, or when they run out of memory. Writing and maintaining error-handling logic in every one of your Lambda functions to handle situations such as API throttling or socket timeouts can be time-intensive and complicated, especially for distributed applications. Embedding this code in each Lambda function creates dependencies between them, and it can be difficult to maintain all of those connections as things change.

To avoid this, and to reduce the amount of error-handling code you write, you can use AWS Step Functions to create a serverless workflow that supports function error handling. Regardless of whether the error is a function exception created by the developer (such as, file not found) or unpredicted (such as, out of memory), you can configure Step Functions to respond with conditional logic based on the type of error that occurred. By separating your workflow logic from your business logic in this way, you can modify how your workflow responds to errors without changing the business logic of your Lambda functions.

What you will accomplish

In this tutorial, you will:

  • design and run a serverless workflow using AWS Step Functions to handle errors
  • create an AWS Lambda function, which will mock calls to a RESTful API and return various response codes and exceptions
  • create a state machine with Retry and Catch capabilities that responds with different logic depending on the exception raised.

Prerequisites

To complete this tutorial, you need an AWS account. If you don't already have an account, follow the Setting Up Your AWS Environment guide for a quick overview.

 

 AWS experience

Intermediate

 Minimum time to complete

60 minutes

 Cost to complete

Free Tier eligible

 Requires

AWS account*


*Accounts created within the past 24 hours might not yet have access to the services required for this tutorial.

 Services used

AWS Step Functions, AWS Lambda, AWS IAM

 Last updated

March 1, 2023

Step 1: Create a Lambda function to mock an API

In this step, you will create a Lambda function that will mock a few basic API interactions. The Lambda function raises exceptions to simulate responses from a fictitious API, depending on the error code that you provide as input in the event parameter.

a. Open the AWS Management Console, so you can keep this step-by-step guide open. When the screen loads, enter your user name and password to get started. Next, enter lambda in the search bar and select Lambda to open the service console.
b. Choose Create function.
c. Leave Author from scratch selected. Next, configure your Lambda function as follows:
For Function name, enter MockAPIFunction.
For Runtime, choose Python 3.9.
For Architecture, choose arm64.
For Role, select Create a new role with basic Lambda permissions.

Choose Create function.
d. On the MockAPIFunction screen, scroll down to the Code source section. In this tutorial, you will create a function that uses the programming model for authoring Lambda functions in Python. In the code window, replace all of the code with the following, then choose Deploy. 
class TooManyRequestsException(Exception): pass
class ServerUnavailableException(Exception): pass
class UnknownException(Exception): pass

def lambda_handler(event, context):
    statuscode = event["statuscode"]    
    if statuscode == "429":
        raise TooManyRequestsException('429 Too Many Requests')
    elif statuscode == "503":
        raise ServerUnavailableException('503 Server Unavailable')
    elif statuscode == "200":
        return '200 OK'
    else:
        raise UnknownException('Unknown error')
e. Make note of the function Amazon Resource Name (ARN). ARNs uniquely identify AWS resources, and help you track and use AWS items and policies across AWS services and API calls. We require an ARN when you need to reference a specific resource from Step Functions. The ARN is found in the Function overview section of the Function page.

Step 2: Create an AWS Identity and Access Management (IAM) role

AWS Step Functions can run code and access other AWS resources (for example, data stored in Amazon S3 buckets). To maintain security, you must grant Step Functions access to these resources using AWS Identity and Access Management (IAM).

a. In another browser window, navigate to the AWS Management Console and enter IAM in the search bar. Select IAM to open the service console.
b. Select Roles, then choose Create role.
c. In the Select trusted entity section, leave the default of AWS service. Under Use case, select Step Functions from the Use cases for other AWS services dropdown, then choose Next.
d. On the Add permissions page, choose Next.
e. On the Name, review, and create page, enter step_functions_basic_execution for Role name and choose Create role.
f. Your new IAM role is created and appears in the list.

Step 3: Create a Step Functions state machine

Now that you’ve created your simple Lambda function that mocks an API response, you can create a Step Functions state machine to call the API and handle exceptions.

In this step, you will use the Step Functions console to create a state machine that uses a Task state with a Retry and Catch field to handle the various API response codes. You will use a Task state to invoke your mock API Lambda function, which will return the API status code you provide as input into your state machine.

a. Open the AWS Step Functions console. On the Create state machine page, select Write your workflow in code. 

b. Next, you will design a state machine that will take different actions depending on the response from your mock API. If the API can’t be reached, the workflow will try again. Retries are a helpful way to address transient errors. The workflow also catches different exceptions thrown by the mock API.

Replace the contents of the state machine definition section with the following code: 
{
 "Comment": "An example of using retry and catch to handle API responses",
 "StartAt": "Call API",
 "States": {
    "Call API": {
        "Type": "Task",
        "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
        "Next" : "OK",
        "Comment": "Catch a 429 (Too many requests) API exception, and resubmit the failed request in a rate-limiting fashion.",
        "Retry" : [ {
            "ErrorEquals": ["TooManyRequestsException" ],
            "IntervalSeconds": 1,
            "MaxAttempts": 2
        } ],
        "Catch": [
            {
                "ErrorEquals": ["TooManyRequestsException"],
                "Next": "Wait and Try Later"
            }, {
                "ErrorEquals": ["ServerUnavailableException"],
                "Next": "Server Unavailable"
            }, {
                "ErrorEquals": ["States.ALL"],
                "Next": "Catch All"
            }
         ]
     },
     "Wait and Try Later": {
        "Type": "Wait",
        "Seconds" : 1,
        "Next" : "Change to 200"
     },
     "Server Unavailable": {
        "Type": "Fail",
        "Error":"ServerUnavailable",
        "Cause": "The server is currently unable to handle the request."
     },
     "Catch All": {
        "Type": "Fail",
        "Cause": "Unknown error!",
        "Error": "An error of unknown type occurred"
     },
     "Change to 200": {
     "Type": "Pass",
     "Result": {
        "statuscode" :"200"
     },
     "Next": "Call API"
     },
     "OK": {
        "Type": "Pass",
        "Result": "The request has succeeded.",
        "End": true
     }
  }
}
Select the recycle button to update the flow diagram:

c. Find the “Resource” line in the “Call API” Task state (line 7). To update this ARN to the ARN of the mock API Lambda function you just created, retrieve the ARN from Step 1e earlier and replace the placeholder.

Choose Next to continue.

d. Fill in the following details:

State machine name: MyAPIStateMachine

Permissions: Select Choose an existing role and select step_functions_basic_execution.

Choose Create state machine.

Step 4: Test your error-handling workflow

To test your error-handling workflow, you will invoke your state machine to call your mock API by providing the error code as input.

a. Choose Start execution.

b. A new execution dialog box appears, where you can enter input for your state machine. You will supply the error code that we want the mock API to return. Replace the existing text with the code below, then choose Start execution:

{
    "statuscode": "200"
}
c. On the Execution details screen, choose Execution input and output to see the input and the final output of the state machine. You can see that the workflow interpreted statuscode 200 as a successful API call.

d. Under Graph view, you can see the execution path of each execution, shown in green in the workflow. Select the Call API Task state and then expand the Input and Output fields in the Step details screen.

You can see that this Task state successfully invoked your mock API Lambda function with the input you provided, and captured the output of that Lambda function, showing “statusCode” : 200.

e. Next, select the OK Task state in the visual workflow. Under Step details you can see that the output of the previous step (the Call API Task state) has been passed as the input to this step. The OK state is a Pass state, which simply passed its input to its output, performing no work. Pass states are useful when constructing and debugging state machines.

Step 5: Inspect the execution of your state machine

a. Scroll to the top of the Execution details screen and select MyAPIStateMachine.
b. Select Start execution again, and this time provide the following input and then choose the  Start execution button. 
{    
    "statuscode": "503"
}

c. In the Execution event history section, expand each execution step to confirm that your workflow behaved as expected. We expected this execution to fail, so don’t be alarmed. You will notice that:

  1. Step Functions captured your Input.
  2. That input was passed to the Call API Task state.
  3. The Call API Task state called your MockAPIFunction using that input.
  4. The MockAPIFunction executed.
  5. The MockAPIFunction failed with a ServerUnavailableException.
  6. The catch statement in your Call API Task state caught that exception.
  7. The catch statement failed the workflow.
  8. Your state machine completed its execution.

d. Next, you will simulate a 429 exception. Scroll to the top of the Execution details screen and select MyAPIStateMachine. Select Start execution, provide the following input, and choose the Start execution button:

{    
    "statuscode": "429"
}

e. Now you will inspect the retry behavior of your workflow. In the Execution event history section, expand each execution step once more to confirm that Step Functions tried calling the MockAPILambda function two more times, both of which failed. At that point, your workflow transitioned to the Wait and Try Later state (shown in the screenshot), in the hopes that the API was just temporarily unresponsive.

Next, the Wait state used brute force to change the response code to 200, and your workflow completed execution successfully. That probably wouldn’t be how you handled a 429 exception in a real application, but we’re keeping things simple for the sake of the tutorial.

f. Run one more instance of your workflow, and this time, provide a random API response that is not handled by your state machine:

{    
    "statuscode": "999"
}

Inspect the execution again using the Execution event history. When complete, select MyAPIStateMachine once more. In the Executions pane, you can see the history of all executions of your workflow, and step into them individually as you like.

Step 6: Clean up resources

In this step, you will terminate resources related to AWS Step Functions and AWS Lambda. Terminating resources that are not actively being used reduces costs and is a best practice. Not terminating your resources can result in a charge.

a. At the top of the AWS Step Functions console window, select State machines.

b. In the State machines window, select MyAPIStateMachine and choose Delete. Confirm the action by choosing Delete state machine in the dialog box. Your state machine will be deleted in a minute or two once Step Functions has confirmed that any in-process executions have completed.

c. Next, you will delete your Lambda functions. Enter lambda into the search bar next to Services, then select Lambda.

d. On the Functions screen, select your MockAPIFunction, choose Actions, and then select Delete. Confirm the deletion by choosing Delete again.
e. Lastly, you will delete your IAM roles. Enter IAM into the search bar next to Services, then select IAM.
f. Select both of the IAM roles that you created for this tutorial, then choose Delete. Confirm the deletion by choosing Yes, delete in the dialog box.
You can now sign out of the AWS Management Console.

Conclusion

Congratulations! You have used AWS Step Functions and AWS Lambda to create an error-handling workflow for a network API. Combining AWS Step Functions with AWS Lambda makes it simple to orchestrate AWS Lambda functions for serverless applications. Step Functions allows you to control complex workflows using Lambda functions without the underlying application managing and orchestrating the state. You can also use Step Functions for microservices orchestration using compute resources such as Amazon EC2 and Amazon ECS.

Was this page helpful?

 Feedback

Next steps

To learn more about Step Functions, explore the following resources.

Follow a self-guided workshop

Walk through interactive modules to learn more about primary Step Functions features.

Dive into the documentation

Find additional tutorials and use cases in the AWS Step Functions Developer Guide. 

Explore FAQs for AWS Step Functions

Get answers to your questions, including how Step Functions works with other AWS services.