How do I invoke a Lambda function using a cross-account Kinesis stream?

Last updated: 2021-12-16

I want to invoke an AWS Lambda function using an Amazon Kinesis stream that's in another AWS account. How do I set that up?

Short description

Lambda doesn't currently support cross-account triggers from Kinesis or any stream-based sources.

As a workaround, you can create a "poller" Lambda function in the same account as the Kinesis stream (account 1). Then, configure the function to invoke a second "processor" Lambda function in the other account (account 2).

Warning: This configuration removes many of the benefits of using Kinesis Data Streams. You can't block records or create sequential ordering within a shard after completing this procedure. It's a best practice to use this workaround only if your application doesn't need those features.

Resolution

Configure a "poller" Lambda function in the same account as the Kinesis stream (account 1)

1.    Create a Lambda function in account 1 that includes an execution role.

Note: You can create the function using the Lambda console, or by building and uploading your own deployment package.

2.    Add the required permissions for your Kinesis stream to the function's execution role.

3.    Configure the Kinesis stream as the function's event source.

Important: When you create the event source mapping, make sure that the Lambda function and the Kinesis stream are in the same account.

Configure a "processor" Lambda function in the second account (account 2)

1.    Create a Lambda function in account 2 that includes an execution role.

2.    Create an AWS Identity and Access Management (IAM) role (invocation role) in account 2.

Note: This invocation role is assumed by the "poller" function in account 1 to invoke the "processor" function in account 2.

3.    Modify the invocation role's policy in account 2 in the following ways:

  • Give the invocation role permissions to invoke (using the lambda:InvokeFunction action) the "processor" Lambda function in account 2.
  • Edit the trust relationship to allow the "poller" Lambda function's execution role in account 1 to assume the invocation role in account 2.

For more information, see Identity-based IAM policies for AWS Lambda. Also, Creating a role to delegate permissions to an AWS service.

Configure the "poller" Lambda function in account 1 to assume the invocation role in account 2

1.    Give the execution role policy in account 1 permissions to call the AssumeRole API and assume the invocation role in account 2. Make sure that you use the sts:AssumeRole action. For more information, see Granting a user permissions to switch roles.

2.    Update the "poller" function in account 1 so that it does the following:

  • Assumes the invocation role in account 2. For instructions, see Switching to an IAM role (AWS API).
  • Passes the input event from Kinesis to the "processor" function in account 2. To have the function pass the input event, instantiate a service client and use the appropriate SDK method to request asynchronous invocation (Event invocation type).
    Note: To determine the appropriate SDK method to call, see the SDK documentation for your runtime.

3.    Configure a dead-letter queue (DLQ) for the function in account 2. This allows you to investigate or retry any missed events if a function error occurs.

Configure a destination for failed-event records

Configure a destination for failed-event records for the event source that's set up for the "poller" function in account 1.

Note: Configuring a destination for failed-event records tells Lambda to send details about discarded records to the destination queue or topic.