Error Handling Patterns in Amazon API Gateway and AWS Lambda

Ryan Green @ryangtweets
Software Development Engineer, API Gateway

A common API design practice is to define an explicit contract for the types of error responses that the API can produce. This allows API consumers to implement a robust error-handling mechanism which may include user feedback or automatic retries, improving the usability and reliability of applications consuming your API.

In addition, well-defined input and output contracts, including error outcomes, allows strongly-typed SDK client generation which further improves the application developer experience. Similarly, your API backend should be prepared to handle the various types of errors that may occur and/or surface them to the client via the API response.

This post discusses some recommended patterns and tips for handling error outcomes in your serverless API built on Amazon API Gateway and AWS Lambda.

HTTP status codes

In HTTP, error status codes are generally divided between client (4xx) and server (5xx) errors. It’s up to your API to determine which errors are appropriate for your application. The table shows some common patterns of basic API errors.

For more information about HTTP server status codes, see RFC2616 section 10.5 on the W3C website.

Routing Lambda function errors to API Gateway HTTP responses

In API Gateway, AWS recommends that you model the various types of HTTP responses that your API method may produce, and define a mapping from the various error outcomes in your backend Lambda implementation to these HTTP responses.

In Lambda, function error messages are always surfaced in the “errorMessage” field in the response. Here’s how it’s populated in the various runtimes:

Node.js (4.3):

exports.handler = function(event, context, callback) {
    callback(new Error("the sky is falling!");
};

Java:

public class LambdaFunctionHandler implements RequestHandler<String, String> {
  @Override
    public String handleRequest(String input, Context context) {
        throw new RuntimeException("the sky is falling!");
   }
}

Python:

def lambda_handler(event, context):
    raise Exception('the sky is falling!')

Each results in the following Lambda response body:

{
  "errorMessage" : "the sky is falling!",
    …
}

The routing of Lambda function errors to HTTP responses in API Gateway is achieved by pattern matching against this “errorMessage” field in the Lambda response. This allows various function errors to be routed to API responses with an appropriate HTTP status code and response body.

The Lambda function must exit with an error in order for the response pattern to be evaluated – it is not possible to “fake” an error response by simply returning an “errorMessage” field in a successful Lambda response.

Note: Lambda functions failing due to a service error, i.e. before the Lambda function code is executed, are not subject to the API Gateway routing mechanism. These types of errors include internal server errors, Lambda function or account throttling, or failure of Lambda to parse the request body. Generally, these types of errors are returned by API Gateway as a 500 response. AWS recommends using CloudWatch Logs to troubleshoot these types of errors.

API Gateway method response and integration response

In API Gateway, the various HTTP responses supported by your method are represented by method responses. These define an HTTP status code as well as a model schema for the expected shape of the payload for the response.

Model schemas are not required on method responses but they enable support for strongly-typed SDK generation. For example, the generated SDKs can unmarshall your API error responses into appropriate exception types which are thrown from the SDK client.

The mapping from a Lambda function error to an API Gateway method responseis defined by an integration response. An integration response defines a selection pattern used to match the Lambda function “errorMessage” and routes it to an associated method response.

Note: API Gateway uses Java pattern-style regexes for response mapping. For more information, see Pattern in the Oracle documentation.

Example:

Lambda function (Node.js 4.3):

exports.handler = (event, context, callback) => {
   callback ("the sky is falling!");
};

Lambda response body:

{
  "errorMessage": "the sky is falling!"
}

API Gateway integration response:

Selection pattern : “the sky is falling!”

Method response : 500

API Gateway response:

Status: 500

Response body:

{
  "errorMessage": "the sky is falling!"
}

In this example, API Gateway returns the Lambda response body verbatim, a.k.a. “passthrough”. It is possible to define mapping templates on the integration response to transform the Lambda response body into a different form for the API Gateway method response. This is useful when you want to format or filter the response seen by the API client.

When a Lambda function completes successfully or if none of the integration response patterns match the error message, API Gateway responds with the default integration response (typically, HTTP status 200). For this reason, it is imperative that you design your integration response patterns such that they capture every possible error outcome from your Lambda function. Because the evaluation order is undefined, it is unadvisable to define a “catch-all” (i.e., “.*”) error pattern which may be evaluated before the default response.

Common patterns for error handling in API Gateway and Lambda

There are many ways to structure your serverless API to handle error outcomes. The following section will identify two successful patterns to consider when designing your API.

Simple prefix-based

This common pattern uses a prefix in the Lambda error message string to route error types.

You would define a static set of prefixes, and create integration responses to capture each and route them to the appropriate method response. An example mapping might look like the following:

Example:

Lambda function (NodeJS):

exports.handler = (event, context, callback) => {
    callback("[BadRequest] Validation error: Missing field 'name'");
};

Lambda output:

{
  "errorMessage": "[BadRequest] Validation error: Missing field 'name'"
}

API Gateway integration response:

Selection pattern: “^\[BadRequest\].*”

Method response: 400

API Gateway response:

Status: 400

Response body:

{
  "errorMessage": "[BadRequest] Validation error: Missing field 'name'"
}

If you don’t want to expose the error prefix to API consumers, you can perform string processing within a mapping template and strip the prefix from the errorMessage field.

Custom error object serialization

Lambda functions can return a custom error object serialized as a JSON string, and fields in this object can be used to route to the appropriate API Gateway method response.

This pattern uses a custom error object with an “httpStatus” field and defines an explicit 1-to-1 mapping from the value of this field to the method response.

An API Gateway mapping template is defined to deserialize the custom error object and build a custom response based on the fields in the Lambda error.

Lambda function (Node.js 4.3):

exports.handler = (event, context, callback) => {
    var myErrorObj = {
        errorType : "InternalServerError",
        httpStatus : 500,
        requestId : context.awsRequestId,
        message : "An unknown error has occurred. Please try again."
    }
    callback(JSON.stringify(myErrorObj));
};

Lambda function (Java):

public class LambdaFunctionHandler implements RequestHandler<String, String> {
  @Override
    public String handleRequest(String input, Context context) {

        Map<String, Object> errorPayload = new HashMap();
        errorPayload.put("errorType", "BadRequest");
        errorPayload.put("httpStatus", 400);
        errorPayload.put("requestId", context.getAwsRequestId());
        errorPayload.put("message", "An unknown error has occurred. Please try again.");
        String message = new ObjectMapper().writeValueAsString(errorPayload);
        
        throw new RuntimeException(message);
    }
}

Note: this example uses Jackson ObjectMapper for JSON serialization. For more information, see ObjectMapper on the FasterXML website.

Lambda output:

{
  "errorMessage": "{\"errorType\":\"InternalServerError\",\"httpStatus\":500,\"requestId\":\"40cd9bf6-0819-11e6-98f3-415848322efb\",\"message\":\"An unknown error has occurred. Please try again.\"}"
}

Integration response:

Selection pattern: “.*httpStatus”:500.*”

Method response: 500

Mapping template:

#set ($errorMessageObj = $util.parseJson($input.path('$.errorMessage')))
{
  "type" : "$errorMessageObj.errorType",
  "message" : "$errorMessageObj.message",
  "request-id" : "$errorMessageObj.requestId"
}

Note: This template makes use of the $util.parseJson() function to parse elements from the custom Lambda error object. For more information, see Accessing the $util Variable.

API Gateway response:

Status: 500

Response body:

{
  "type": "InternalServerError",
  "message": " An unknown error has occurred. Please try again.",
  "request-id": "e308b7b7-081a-11e6-9ab9-117c7feffb09"
}

This is a full Swagger example of the custom error object serialization pattern. This can be imported directly into API Gateway for testing or as a starting point for your API.

{
  "swagger": "2.0",
  "info": {
    "version": "2016-04-21T23:52:49Z",
    "title": "Best practices for API error responses with API Gateway and Lambda"
  },
  "schemes": [
    "https"
  ],
  "paths": {
    "/lambda": {
      "get": {
        "consumes": [
          "application/json"
        ],
        "produces": [
          "application/json"
        ],
        "parameters": [
          {
            "name": "status",
            "in": "query",
            "required": true,
            "type": "string"
          }
        ],
        "responses": {
          "200": {
            "description": "200 response",
            "schema": {
              "$ref": "#/definitions/Empty"
            }
          },
          "400": {
            "description": "400 response",
            "schema": {
              "$ref": "#/definitions/Error"
            }
          },
          "403": {
            "description": "403 response",
            "schema": {
              "$ref": "#/definitions/Error"
            }
          },
          "404": {
            "description": "404 response",
            "schema": {
              "$ref": "#/definitions/Error"
            }
          },
          "500": {
            "description": "500 response",
            "schema": {
              "$ref": "#/definitions/Error"
            }
          }
        },
        "x-amazon-apigateway-integration": {
          "responses": {
            "default": {
              "statusCode": "200"
            },
            ".*httpStatus\\\":404.*": {
              "statusCode": "404",
              "responseTemplates": {
                "application/json": "#set ($errorMessageObj = $util.parseJson($input.path('$.errorMessage')))\n#set ($bodyObj = $util.parseJson($input.body))\n{\n  \"type\" : \"$errorMessageObj.errorType\",\n  \"message\" : \"$errorMessageObj.message\",\n  \"request-id\" : \"$errorMessageObj.requestId\"\n}"
              }
            },
            ".*httpStatus\\\":403.*": {
              "statusCode": "403",
              "responseTemplates": {
                "application/json": "#set ($errorMessageObj = $util.parseJson($input.path('$.errorMessage')))\n#set ($bodyObj = $util.parseJson($input.body))\n{\n  \"type\" : \"$errorMessageObj.errorType\",\n  \"message\" : \"$errorMessageObj.message\",\n  \"request-id\" : \"$errorMessageObj.requestId\"\n}"
              }
            },
            ".*httpStatus\\\":400.*": {
              "statusCode": "400",
              "responseTemplates": {
                "application/json": "#set ($errorMessageObj = $util.parseJson($input.path('$.errorMessage')))\n#set ($bodyObj = $util.parseJson($input.body))\n{\n  \"type\" : \"$errorMessageObj.errorType\",\n  \"message\" : \"$errorMessageObj.message\",\n  \"request-id\" : \"$errorMessageObj.requestId\"\n}"
              }
            },
            ".*httpStatus\\\":500.*": {
              "statusCode": "500",
              "responseTemplates": {
                "application/json": "#set ($errorMessageObj = $util.parseJson($input.path('$.errorMessage')))\n#set ($bodyObj = $util.parseJson($input.body))\n{\n  \"type\" : \"$errorMessageObj.errorType\",\n  \"message\" : \"$errorMessageObj.message\",\n  \"request-id\" : \"$errorMessageObj.requestId\"\n}"
              }
            }
          },
          "httpMethod": "POST",
          "requestTemplates": {
            "application/json": "{\"failureStatus\" : $input.params('status')\n}"
          },
          "uri": "arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/[MY_FUNCTION_ARN]/invocations",
          "type": "aws"
        }
      }
    }
  },
  "definitions": {
    "Empty": {
      "type": "object"
    },
    "Error": {
      "type": "object",
      "properties": {
        "message": {
          "type": "string"
        },
        "type": {
          "type": "string"
        },
        "request-id": {
          "type": "string"
        }
      }
    }
  }
}

Conclusion

There are many ways to represent errors in your API. While API Gateway and Lambda provide the basic building blocks, it is helpful to follow some best practices when designing your API. This post highlights a few successful patterns that we have identified but we look forward to seeing other patterns emerge from our serverless API users.