AWS Compute Blog

Introducing the Amazon EventBridge service integration for AWS Step Functions

This post is courtesy of Stephen Liedig, Sr Serverless Specialist SA.

AWS Step Functions now integrates with Amazon EventBridge to provide a simpler solution for producing events during a workflow.

Step Functions allows you to build resilient serverless orchestration workflows with AWS services such as AWS Lambda, Amazon SNS, Amazon DynamoDB, and more. Step Functions provides a history of executions for a given state machine in the AWS Management Console or with Amazon CloudWatch Logs.

EventBridge allows you to route events between AWS services, integrated software as a service (SaaS) applications, and your own applications. Event producers publish events onto an event bus, which uses rules to determine where to send those events. The rules can specify one or more targets, which can be other AWS services or API Destinations. This model makes it easier to develop scalable, distributed serverless applications by handling event routing and filtering.

What’s new?

The new Step Functions integration with EventBridge provides an additional resource type – arn:aws:states:::events:putEvents. This is used with both standard and Express Workflows. It allows customers to publish events to a specified event bus directly from a workflow, using one of two integration patterns:

  1. Request-Response: calling a service and letting Step Functions progress to the next state immediately after it receives an HTTP response. This pattern is supported by standard and Express Workflows. 
    Send an EventBridge custom event:
  Type: Task
  Resource: 'arn:aws:states:::events:putEvents'
  Parameters:
    Entries:
      - Detail:
          Message: 'Hello from Step Functions!'
        DetailType: MyDetailType
        EventBusName: MyEventBusName
        Source: MySource
  Next: NEXT_STATE
  2. Wait-for-callback: calling a service with a task token and having Step Functions wait until that token is returned with a payload. This pattern is supported by Standard Workflows. 
    Send an EventBridge custom event:
  Type: Task
  Resource: 'arn:aws:states:::events:putEvents.waitForTaskToken'
  Parameters:
    Entries:
      - Detail:
          Message: 'Hello from Step Functions!'
          TaskToken.$: $$.Task.Token
        DetailType: MyDetailType
        EventBusName: MyEventBusName
        Source: MySource
  Next: NEXT_STATE

The new integration is configured with the following Amazon States Language parameter fields:

  • Detail: A valid string or JSON object
  • DetailType: Free-form string used to decide what fields to expect in the event detail.
  • EventBusName: The name or ARN of the event bus to receive the event. Only the rules that are associated with this event bus are used to match the event. If you omit this, the default event bus is used. The ARN can also be that of an event bus in another account.
  • Source: The source of the event.
  • Time: The event timestamp (optional)
  • Resources: This JSON array contains ARNs that identify resources that are involved in the event (optional).

Refer to the documentation for more information on EventBridge fields and concepts.

Getting started

This integration can be configured using AWS Serverless Application Model (AWS SAM), the AWS Command Line Interface (AWS CLI), AWS CloudFormation, or from within the AWS Management Console.

To get started using the AWS Management Console:

  1. Navigate to the Step Functions console.
  2. Choose Run a sample project and choose Send a custom event to EventBridge.

    Run a sample project

    Run a sample project

  3. The Definition section shows the ASL that makes up the example workflow. The following example shows the new EventBridge resource and its parameters:

    Example definition

    Example definition

  4. Review example Definition, then choose Next.
  5. Choose Deploy resources.

This deploys a standard Step Functions workflow and an EventBridge event bus and a rule that sends events from Step Functions to three targets: a AWS Lambda function, an Amazon SNS topic, and an Amazon SQS queue.

It deploys an IAM role with the required permissions to allow Step Functions to send messages to the event bus. Roles and resource policies are also created to allow EventBridge to send events to the respective targets.

Running the workflow

  1. Choose the newly created state machine from the Step Functions console.
  2. Choose Start execution.
  3. Clear the input field.
  4. Choose Start execution.
  5. Choose the Send a custom event step, then choose the Step output tab.This shows if the events are successfully or unsuccessfully delivered to EventBridge.

    Set output configuration

    Set output configuration

Access control

The EventBridge integration supports AWS Identity and Access Management (IAM) authentication and authorization. This includes IAM roles, policies, and tags. AWS IAM roles and policies offer flexible and robust access controls that can be applied to create or access EventBridge resources. Tag-based access control allows you to set more fine-grained access control for all EventBridge resources. Specify tag key-value pairs to categorize EventBridge resources by environment or other criteria.

EventBridge resource policies are JSON policy documents that control whether a specified principal (typically an IAM user or an AWS account) can send events or create rules on an event bus. Resource policies can grant access to Step Functions to send events to an event bus. This could be for workflows in a different AWS account or only for specified state machine. For more information, see Introducing cross-Region event routing with Amazon EventBridge.

To configure access control for the EventBridge integration, Step Functions must assume the role with the following policy:

{
  "Statement": [
    {
      "Action": "events:PutEvents",
      "Resource": "arn:aws:events:[REGION]:[ACCOUNT]:event-bus/[EVENT BUS NAME]",
      "Effect": "Allow"
    }
  ]
}

 

Microservices example

Know your customer (KYC) guidelines are typical policies and standards in many financial institutions that involve identifying customers and establishing a risk profile. In this example, I show how to use Step Functions to model a KYC workflow and use it to integrate two business domains: Accounts and Customer Service.

The process is event-driven, promoting the concepts of loose coupling, isolation, and autonomy. It uses a low code integration approach via the new EventBridge service integration.

Know You Customer (KYC) workflow

Know You Customer (KYC) workflow

  1. The event flow begins with the accounts system, which omits an event that a new account has been requested.
  2. This event triggers the workflow within the KYC service.
  3. The KYC process verifies a customer’s identity and risk profile. When the identity check is completed, it emits the “Identity check completed” event.
  4. KYC process publishes two events to signal approval or rejection of the new account request, dependent on the outcome of the risk profile assessment.
  5. These events are consumed by the accounts and customer service domains. Each has rules defined on a central event bus to process the results of the KYC processing.

Using the new service integrations for EventBridge, developers can explicitly model events into state machines. This can happen at any stage of the state machine execution and is a useful way to signal state transitions to interested consumers. For long running processes, it replaces the need for consumers to know the state machine execution ID to query the execution history with the DescribeExecution API call. Instead, consumers can subscribe to events emitted by the state machine and remain decoupled from the implementation.

If you need the execution ID from the state machine, this is provided by the service integration. For example, the “New account approved” state defined previously sends an event that contains the ARN of the state machine and the state machine execution in the Resource field. Step Functions adds this automatically with the time of the state execution and the account and Region it was invoked in.

{
  "version": "0",
  "id": "005fc46c-f5d7-a074-06a2-0032c07529d7",
  "detail-type": "New account approved",
  "source": "com.aws.kyc",
  "account": "111122223333",
  "time": "2021-05-06T03:08:23Z",
  "region": "ca-central-1",
  "resources": [
    "arn:aws:states:ca-central-1:111122223333:stateMachine:KycStateMachine-jUkKkPuut800",
    "arn:aws:states:ca-central-1:111122223333:execution:KycStateMachine-jUkKkPuut800:06bce19f-8b25-2982-a704-ad78cc1a29e5_afe130b6-0c15-e61f-90be-a27f38511a15"
  ],
  "detail": {
    "customerAddress": "123 Any Street, Any Town, USA",
    "agencyChecked": true,
    "indentityChecked": true,
    "customerName": "Nikki Wolf",
    "checksPassed": true
  }
}

To run the application

  1. Clone the GitHub repository: 
    git clone https://github.com/aws-samples/aws-stepfunctions-examples.git
cd ./sam/app-kyc-with-eventbridge
  2. Build and deploy the application using AWS SAM CLI, accepting the default parameter inputs: 
    sam build
sam deploy -g
    SAM deploy output

    SAM deploy output

    This deploys several resources defined in the template.yaml file, including a Step Functions standard workflow, EventBridge event bus, and Amazon CloudWatch log groups for the EventBridge rule targets.

Testing the application

  1. Manually send some events to the event bus, emulating a new account requested event that initiates the KYC workflow: 
    aws events put-events —entries file://put_events.json

The put_events.json file contains the events that are sent to the central event bus that start the state machine execution. Ensure that the name in the EventBusName field matches the name of the event bus created by the AWS SAM deployment.

{
  "Source": "com.aws.accounts",
  "Detail": "{\"customerName\": \"Nikki Wolf\", \"customerAddress\": \"123 Any Street, Any Town, USA\" }",
  "DetailType": "New account requested",
  "EventBusName": "CentralEventBus"
}

The response shows that the events are sent successfully:

{
  "FailedEntryCount": 0,
  "Entries": [
    {
      "EventId": "9f6d6147-7f77-dfe8-f3b9-79b38833d258"
    },
    {
      "EventId": "db695c48-da98-69a8-f82a-ad3cc04e07bd"
    }
  ]
}

The state machine is invoked by a rule on the central event bus. This rule is defined in the AWS SAM template as a state machine event source for the serverless state machine resource.

The workflow is defined in a separate file (/statemachine/kyc.asl.json). The DefinitionSubstitutions field is used to pass template parameters to the workflow definition. In this case, the name of the Central event bus is passed as a parameter to the CentralEventBusName substitution variable. This substitution variable is passed to the state machine definition file, and is used by the EventBridge service integration states defined in the state machine to set the name of the event bus to send events to.

KycStateMachine:
  Type: AWS::Serverless::StateMachine
  Properties:
    DefinitionUri: statemachine/kyc.asl.yaml
    DefinitionSubstitutions:
      CentralEventBusName: !Ref CentralEventBus
    Policies:
      - AWSXRayDaemonWriteAccess
      - EventBridgePutEventsPolicy:
        EventBusName: !Ref CentralEventBus
      - Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Action:
              - 'logs:CreateLogDelivery'
              - 'logs:GetLogDelivery'
              - 'logs:UpdateLogDelivery'
              - 'logs:DeleteLogDelivery'
              - 'logs:ListLogDeliveries'
              - 'logs:PutResourcePolicy'
              - 'logs:DescribeResourcePolicies'
              - 'logs:DescribeLogGroups'
            Resource: !GetAtt  StateMachinesLogGroup.Arn
    Logging:
      Destinations:
        - CloudWatchLogsLogGroup:
          LogGroupArn: !GetAtt  StateMachinesLogGroup.Arn
    Tracing:
      Enabled: True
    Events:
      EBRule:
        Type: EventBridgeRule
        Properties:
          EventBusName: !Ref CentralEventBus
          InputPath: $.detail
          Pattern:
            source:
              - com.aws.accounts
            detail-type:
              - New account requested
            account:
              - !Ref AWS::AccountId

 

The definition itself contains 3 new arn:aws:states:::events:putEvents resources that  declares the EventBridge service integration type.

The parameters object holds the required fields to configure the service integration. The Entries define one or more events that are sent to the event bus.  The EventBusName values are provided by the DefinitionsSubstitutions field in the AWS SAM template. The Detail input is defined dynamically using Amazon States Language. The “.$” at the end of the field name Detail specifies that the parameter uses a path to reference a JSON node in the input.

Identity check completed:
  Comment: >-
    Publish event when identity check has been completed
  Type: Task
  Resource: 'arn:aws:states:::events:putEvents'
  Parameters:
    Entries:
      - Detail.$: $
        DetailType: Identity check completed
        EventBusName: '${CentralEventBusName}'
        Source: com.aws.kyc
  Next: Verify risk profile
  Retry:
    - ErrorEquals:
        - States.ALL
      IntervalSeconds: 1
      BackoffRate: 2
      MaxAttempts: 2

Cleanup

Once you have completed this tutorial, be sure to remove the stack from CloudFormation. From the CloudFormation console, select your application, and then choose Delete.

Conclusion

The Step Functions integration with EventBridge provides customers with the ability to call PutEvents directly from a Step Functions workflow.

The ability to explicitly model and publish events inside workflows reduces the coupling between orchestration logic and other distributed components or services. It also replaces the need for Lambda functions to call the EventBridge APIs.

This feature is available in all Regions where both Step Functions and EventBridge are available. View the AWS Regions table to learn more. For pricing information, see Step Functions pricing. Normal Service Quotas for EventBridge and service limits of Step Functions apply.

For more serverless learning resources, visit Serverless Land.