A Guide to Developing Serverless Ecommerce Workflows for commercetools with AWS Lambda

Now more than ever, consumers want to discover and purchase their favorite products online. To meet this increased demand, retailers need to improve the performance of their ecommerce applications.

In this blog post, we’ll guide web developers through the process of integrating commercetools’ Merchant Center with AWS Lambda to develop a serverless ecommerce workflow.

Solution Overview

In this common use case, product-related data is synchronized with an external data store to automate processes like email notifications and managing inventory capacity. This solution connects the product data from a publish event to a custom AWS Lambda workflow to demonstrate the capabilities of AWS to your commercetools workflow. You will deploy an AWS CloudFormation stack that uses serverless services, such as Amazon Simple Notification Service (Amazon SNS), AWS Lambda, and Amazon DynamoDB. The commercetools subscription uses AWS Identity and Access Management (IAM) user credentials to securely publish messages to Amazon SNS.

Figure 1: Solution architecture

Message Format for commercetools

Before you integrate the deployed resources with commercetools, you should understand the format of commercetools messages and how the AWS Lambda function works.

The following snippet is a sample commercetools message payload for a ProductPublished event. For brevity, we have omitted the extraneous fields.

{
    "notificationType": "Message",
    "projectKey": "<project_key>",
    "id": "<message_id>",
    "version": 1,
    "sequenceNumber": 5,
    "resource": {
        "typeId": "product",
        "id": "<resource_id>"
    },
    "resourceVersion": 11,
    "resourceUserProvidedIdentifiers": {
        "slug": {
            "en-US": "test-product"
        }
    },
    "type": "ProductPublished",
    "productProjection": {
        "id": "<product_id>",
        "version": 11,
        "productType": {
            "typeId": "product-type",
            "id": "<product_type_id>"
        },
        "name": {
            "en-US": "Test Products"
        },
        "description": {
            "en-US": "This is an amazing product!"
        },
        ...
        "slug": {
            "en-US": "test-product"
        },
        ...
        "masterVariant": {
            "id": 1,
            "sku": "5877823",
            "key": "5877823",
            ...
        },
        "variants": [],
        "searchKeywords": {},
        "hasStagedChanges": false,
        "published": true,
        "createdAt": "2021-04-20T16:04:32.930Z",
        "lastModifiedAt": "2021-04-20T20:28:29.866Z"
    },
    "scope": "All",
    "createdAt": "2021-04-20T20:28:29.866Z",
    "lastModifiedAt": "2021-04-20T20:28:29.866Z"
}

Note these AWS Lambda function fields:

type – The message type that represents a change or action performed on a resource.
productProjection.id – The ID assigned to the product by commercetools.
productProjection.name – The display name of the product across all supported languages.
productProjection.description – The description of the product across all supported languages.
productProjection.version – The version assigned to the product entity by commercetools.

For more information about the commercetools message format and fields, see Message Subscription Payload and the productProjection section of Representations in the commercetools documentation.

AWS Lambda Function Examination

The following Node.js code is deployed in the AWS Lambda function definition section of the AWS CloudFormation template. The function performs the following actions:

For each received event record, parses the event record message object. (This message object contains the commercetools event message.)
Filters the message object based on a message type of ProductPublished and maps it to the product projection payload.
The storeProducts () function saves specific fields (product ID, name, description, version, and so on) to an Amazon DynamoDB table and then returns the stored products.

const AWS = require('aws-sdk');
const documentClient = new AWS.DynamoDB.DocumentClient();

exports.handler = async (event) => {
  console.log('event', JSON.stringify(event));

  const publishedProducts = event.Records
    .map(record => JSON.parse(record.Sns.Message))
    .filter(message => message.type === 'ProductPublished')
    .map(message => message.productProjection);

  const storedProducts = await storeProducts(publishedProducts);

  return {
    statusCode: 200,
    body: storedProducts
  };
};

const storeProducts = async (products) => {
  if (products.length === 0) {
    return [];
  }

  const productItems = products.map(product => {
    return {
      id: product.id,
      name: product.name['en-US'],
      description: product.description['en-US'],
      version: product.version
    };
  });

  // Chunk the productItems array for batch writes
  const chunk = 25;
  for (let i = 0, j = productItems.length; i < j; i += chunk) {
    const params = {
      RequestItems: {
        'CTProducts': productItems.slice(i, i + chunk).map(productItem => {
          return {
            PutRequest: {
              Item: productItem
            }
          };
        })
      }
    };

    // TODO: Be sure to handle UnprocessedItems appropriately for your use case
    await documentClient.batchWrite(params).promise();
  }

  return productItems;
};

Walkthrough

In this walkthrough, you’ll follow these steps to create a serverless workflow for commercetools with AWS resources:

Deploy the AWS CloudFormation template.
Configure a commercetools product subscription.
Create and publish a commercetools product.
Review commercetools messages in Amazon CloudWatch Logs and Amazon DynamoDB.
Delete the CloudFormation stack.

Prerequisites

If you’re new to commercetools or just need a refresher, follow the steps in the Getting Started with commercetools tutorial.

To complete the steps in this walkthrough, you’ll need the following:

An AWS account.
A commercetools account. If you do not have an account, you can start a free trial.
The commercetools Postman collection that is configured with API client
Familiarity or previous experience with:
- Deploying an AWS CloudFormation template.
- Creating an AWS Lambda function.
- Reading and writing in Amazon DynamoDB tables.
- Using the AWS SDK for JavaScript with Node.js.

Deploy the CloudFormation Template

Use the one-click AWS CloudFormation deployment feature to deploy your AWS resources:

AWS will provision the following resources through the AWS CloudFormation deployment:

An SNS topic to handle incoming messages from commercetools.
An IAM user, which is used by commercetools to securely publish messages to Amazon SNS.
A Lambda function to process SNS messages and write them to DynamoDB.
A DynamoDB table to store commercetools product information.
An IAM role and permissions required for Amazon SNS, AWS Lambda, and Amazon DynamoDB interactions.

Make a note of the AWS CloudFormation stack output values in the AWS CloudFormation console. You will need the CTEventsTopicArn, CTSubscriptionUserAccessKey, and CTSubscriptionUserAccessSecret values later.

Figure 2: Sample AWS CloudFormation stack output that includes the topic ARN, IAM user access key ID, and secret access key link

To retrieve the access secret value, navigate to the commercetoolsPublishUserAccessSecret link in the CloudFormation stack output and copy the access secret in the Value field for the SSM parameter.

Configure a commercetools Product Subscription

Now, you’ll create a product subscription that publishes messages to the SNS topic whenever an action, such as a product publish, occurs in the Merchant Center. You will use the commercetools’ Postman collection to configure and create a subscription for product-specific actions.

To create the commercetools subscription in Postman, follow these steps:

Open the commercetools collection in Postman.
Navigate to the Products, Subscriptions subfolder, and open the Create Subscription POST request.
In the Body tab for the request, make the following changes to the destination JSON:
1. Change the type value to SNS.
2. Remove the uri field and value.
3. Add a topicArn field with the value of CTEventsTopicArn from the previous step.
4. Add an accessKey field with the value of CTSubscriptionUserAccessKey from the previous step.
5. Add an accessSecret field with the value of CTSubscriptionUserAccessSecret from the previous step.
Note that the resourceTypeId for the messages field is already set to product.
(Optional) Update the key field to a more meaningful or unique value.
Verify the request body is similar to the following code:

{
    "destination": {
        "type": "SNS",
        "topicArn": "<your CTEventsTopicArn>",
        "accessKey": "<your CTSubscriptionUserAccessKey>",
        "accessSecret": "<your CTSubscriptionUserAccessSecret>"
    },
    "messages": [
        {
            "resourceTypeId": "product",
            "types": []
        }
    ],
    "key": "test-queue"
}

Choose Send.

If you received a 201 created status, your commercetools subscription is ready to go. If you received an error, confirm your client API key is configured correctly and that you invoked the Obtain Access Token POST request under the Authorization subfolder.

For information about other options available for commercetools subscriptions, see the Create a Subscription in the commercetools documentation.

Create and Publish a commercetools Product

Now that you’ve created the commercetools subscription and configured it with the SNS topic, you are ready to send product changes over to AWS.

Sign in to your commercetools Merchant Center account and follow the instructions to create and publish a sample product. This product is contained in the publish event, which is sent to the SNS topic.

In this example, the product name and description should be set with en-US as the language.

Now that you’ve published a product change, in the next section, you’ll examine how the AWS resources have captured and published product information.

Review commercetools Messages in Amazon CloudWatch Logs and Amazon DynamoDB

To review how the published message flows through your deployed resources, follow these steps to inspect Amazon CloudWatch Logs and Amazon DynamoDB table values:

In the left navigation pane of the AWS Lambda console, choose Functions, and then open the HandleCTEvents
On the Monitor tab, choose the View Logs in CloudWatch
On the tab that opens with the log streams associated with the HandleCTEvents function, choose the most recent log stream based on the last event time.
To inspect the event JSON message received in AWS Lambda, expand the most recent log statement that contains the INFO event line.
- Note that the Sns.TopicArn field matches the SNS topic ARN from the CloudFormation stack output.
- The Sns.Message field contains a string-formatted JSON payload of the commercetools subscription message.

Figure 3: A CloudWatch log for an SNS topic message processed by AWS Lambda

In the left navigation pane of the Amazon DynamoDB console, choose Tables, and then open the CTProducts
In Items preview, you should see a record for the product that was published from commercetools in the previous step. The id column displays the unique ID that commercetools assigned to the product. The name and description columns display the latest values from commercetools. The version column displays the version of the product, which is incremented when a product attribute is changed in commercetools.

Figure 4: Published changes for a product from commercetools propagates to a DynamoDB table

If you make another change to the product from the Merchant Center and publish it, you can examine the updated changes in the CTProducts table along with a new version.

Congratulations! You’ve verified that product events are processed in the AWS resources!

Clean Up

To avoid additional charges to your account, delete the CloudFormation stack.

Conclusion

I hope you found this post helpful to securely integrate commercetools with AWS resources. While the example we outlined is a simple use case with a handful of AWS services, you can integrate and customize the use case according to your needs.

With commercetools, you can also personalize your ecommerce application to meet the needs of your customers.

Here are some other blog posts you might find helpful:

For more examples about how to use AWS ecommerce solutions, check out A Perfect Marriage: Ecommerce on AWS.
To learn more about serverless architecting with AWS Lambda, check out Serverless Architectures with AWS Lambda: Overview and Best Practices.
For more information about AWS tools for your retail store, check out Recapping re:Invent: What’s New in Retail.

If you’re ready to build a serverless ecommerce solution, AWS is here to help you. Contact your account team or visit Retail on AWS today to get started.