AWS Cloud Operations Blog

How to update AWS Service Catalog provisioned products to new product versions and report changes using AWS Step Functions, AWS Lambda, and Amazon Athena

AWS Service Catalog enables organizations to centrally manage commonly-deployed AWS services and provisioned software products. It also helps you achieve consistent governance and compliance requirements, while it empowers users to self-serve and quickly deploy the approved AWS services they need.

Customers that I talk to want to have a mechanism to keep previously provisioned products up to date after introducing a new version of the product to reduce the configuration drift. For example, let’s say that you have a new AWS Service Catalog version of an Amazon Relational Database Service (Amazon RDS) database that you changed to a newer version of database engine. You want to be able to update the existing RDS database instances that were launched through AWS Service Catalog without having to manually change the configurations of the RDS instances.

In this blog post, I’ll walk you through a solution that enables you to configure products in AWS Service Catalog and automatically update previously provisioned products to keep them up to date with the newly introduced version. I’ll also show you how you can track version updates and status using Amazon Athena to query log files from the solution. The solution works with AWS Service Catalog products that are provisioned in the same AWS account in which the solution is deployed.

Solution overview

The solution uses combination of the following AWS services: AWS CloudFormation, Amazon CloudWatch, AWS Lambda, and AWS Step Functions. I’ll be using Amazon CloudWatch Events to trigger an AWS Lambda function based on a scheduled time. CloudWatch Events can trigger various services on AWS based on events that occur. It can also be configured to self-trigger at scheduled times. AWS Lambda lets you run code without provisioning or managing servers. AWS Step Functions enables you to coordinate multiple AWS services into a serverless workflow.

Here is the architecture diagram of the solution.

    1. The solution explained in this post uses a CloudWatch Events rule to periodically trigger an AWS Lambda function. The default schedule for the CloudWatch event in the solution is to run at 10:00PM (UTC) every Thursday. You can adjust this schedule during the deployment of the AWS CloudFormation template, which I will explain in the next section.
    2. The Lambda function scans for provisioned products in the AWS account and Region in which the solution is deployed. For each provisioned product, the Lambda function verifies whether version of the provisioned product is the latest version defined in AWS Service Catalog. It also verifies that the AWS Service Catalog administrator has enabled the product to be automatically updated by the solution. If you have a hub and spoke model set up, you need to deploy this solution in each spoke account in the appropriate Regions.
    3. If the conditions are met, then the Lambda function triggers AWS Step Functions state machine execution to update the provisioned product. The updated provisioned product operation of AWS Service Catalog triggers the CloudFormation stack update operation. This operation can take some time to complete, depending on the AWS resources that are being updated. By using the AWS Step Functions state machine, the solution can run and track the status of the multiple update operations at the same time independently outside of the Lambda function. The following diagram illustrates the state machine definition.

  1. The Lambda functions generate log files in JSON format and are stored in an Amazon S3 bucket. The log file contain the status of the CloudFormation stack updates and error messages if the update operations had failed.

Warning

CloudFormation stack updates can cause AWS resources to be terminated based on the changes made in the template. To avoid this situation, click here to learn more about the update behaviors of stack resources and versions for upgrades of Amazon RDS and Amazon ElastiCache (Memcache and Redis).

Let’s try it out!

Step 1 – Deploy CloudFormation template

You can deploy the solution by using the CloudFormation template provided as part of this blog post. The key AWS resources that the CloudFormation template will create in your account are:

  • Amazon CloudWatch Events rule
  • AWS Lambda functions
  • AWS Service Catalog tag option
  • AWS Step Functions state machine
  • IAM roles and policies

To deploy the CloudFormation template:

  1. Sign in to the AWS Management Console and go to the AWS CloudFormation console.
  2. Next, launch the CloudFormation stack by choosing the following Launch Stack link.

  1. Choose Next.
  2. On the Specify Details page, enter the stack name and input parameters for CloudFormation stack.

  1. Log Configuration

    • logS3Bucket – Amazon S3 bucket name for log files generated from Lambda functions
    • logS3Key – Amazon S3 object key used as folder name under logS3Bucket for the log files (default = ServiceCatalogLogs)

    Service Catalog TagOption Configuration

    • tagOptionKey – TagOption key for enabling automatic update of a provisioned AWS Service Catalog product to the latest version (default = AutoUpdateProvisionedProduct)
    • tagOptionValue – TagOption value for enabling automatic update of a provisioned AWS Service Catalog product to the latest version (default = true)

    CloudWatch Event Rule Configuration

    • cronExpressionForEventRule – Cron expression of schedule for triggering CloudWatch Event Rule. As stated earlier, the default schedule is at 10:00PM (UTC) every Thursday, you can adjust this to match your requirement.
  1. Choose Next.
  2. On the Options page, choose Next.
  3. On the Review page, select the check box displayed next to the following message.
    • I acknowledge that AWS CloudFormation might create IAM resources with custom names.
  1. Choose Create. The CloudFormation template creates the AWS resources for the solution.

Note: If you encounter the TagOptionNotMigratedException exception during the stack creation, this is because the TagOptions migration process has not been performed for this account. Please use the console to perform the migration process before retrying the operation.

  1. Sign in to the AWS Management Console with the IAM user that has AWS Service Catalog Administrator permission and go to the AWS Service Catalog console.
  2. Under the Admin section, select TagOption Library.
  3. On the console, you will see the following error message:

  1. Choose Copy Tags and the TagOption migration process will start.
  2. After the migration is completed, you will receive the following message:

  1. Re-deploy the CloudFormation template, and this time it should succeed if it had failed before due to the TagOptionNotMigratedException error.

Step 2 – Configure TagOption

After the completion of the CloudFormation stack creation, you can use the AWS Service Catalog Tag Option to enable automatic update of provisioned AWS Service Catalog products. Because not all AWS Service Catalog products will require automatic update when there is a new version introduced, TagOption gives you the flexibility to selectively apply this solution.

  1. Sign in to the AWS Management Console with IAM user that has AWS Service Catalog Administrator permission and go to the AWS Service Catalog console.
  2. Under the Admin section, choose Products List.
  3. On the Admin products page, choose the product that you want to enable for automatic update of provisioned products.
  4. On the Admin products – Admin product details page, choose TagOptions.

  1. Choose Add TagOption.
  2. On the Add / associate TagOptions pop-up window, select the check box corresponding to the key and value that you have entered as the CloudFormation stack input parameter in Step 1 and choose Save.

  1. Repeat the steps in this section for other AWS Service Catalog products as needed. For products in AWS Service Catalog for which you don’t want to enable automatic update, there is no need to associate the TagOption to the products.

If the AWS Service Catalog product to which you have applied the TagOption is distributed to multiple portfolios, the automatic update will be done regardless of the portfolio to which the product belongs. If you are using the hub-and-spoke model, the TagOption must be associated with the product at the spoke level.

Step 3 – Configure AWS Lambda role access to portfolios

  1. Sign in to the AWS Management Console with the IAM user that has AWS Service Catalog Administrator permission and go to the AWS Service Catalog console.
  2. Under the Admin section, choose Portfolios List.
  3. On the Portfolios list page, choose the portfolio that contains products that you have enabled for automatic update of provisioned products.
  4. On the Portfolios list – Portfolio details page, choose Users, groups and roles.
  5. Choose Add User, Group or Role.
  6. On the Add users, groups and roles access for [portfolio name] page, select the roles tab and select the check box for ServiceCatalogMonitorRole, ServiceCatalogMonitorUpdateRole, ServiceCatalogMonitorCheckUpdateRole, ServiceCatalogMonitorLogRole role and choose Add Access. This role is created by the solution CloudFormation stack. By giving the role access to the portfolio, it allows the Lambda functions to retrieve necessary Service Catalog definitions related to the portfolio and products.
  7. Repeat the steps in this section for other Service Catalog portfolios as needed.

Step 4 – Run the Lambda function

If you want to see the solution in action without waiting for the triggering of the CloudWatch Event rule, you can manually trigger the main Lambda function by doing the following:

  1. Sign in to the AWS Management Console and go to the AWS Lambda console.
  2. Choose the ServiceCatalogMonitor-Main function.
  3. On the ServiceCatalogMonitor-Main page, choose Test.
  4. On the Configure test event page, choose Create new test event and in the Event template, leave the default Hello World option. Enter an Event name and choose Create.

The Lambda functions deployed in this solution generate log files in JSON format so you can see the provisioned products that are scanned, whether the provisioned products are provisioned from the latest version, and the status of the product update (if automatic update is configured). Amazon Athena provides an interactive query service that makes it easy to analyze data in Amazon Simple Storage Service (Amazon S3) using standard SQL. In the next step, we will explore how we can use Amazon Athena to query the log files.

Step 5 – Viewing log files with Amazon Athena (optional)

The log file generated by the Lambda function is in the following format.

{
    "AutoUpdateProvisionedProduct": “”,
    "Errors": "",
    "ProvisionedProductName": "",
    "AccountNo": "",
    "ProductId": "",
    "LaunchedBy": "",
    "ProductName": "",
    "StackName": "",
    "LatestVersionName": "",
    "ScannedTime": "",
    "ProvisionedVersion": "",
    "UpdateTimestamp": "",
    "LatestVersionId": "",
    "UpdateStatus": "",
    "ProvisionedProductId": "",
    "ProvisionedVersionName": ""
}

You can configure an Amazon Athena table to use the standard SQL query to query the data in log files by doing the following:

    1. Sign in to the AWS Management Console and go to the Amazon Athena console.
    2. Choose an existing database or create a new database.
    3. Update the Amazon S3 location in the following DDL and create a table (the format of LOCATION is “s3://< logS3Bucket>/<logS3Key>”):
CREATE EXTERNAL TABLE `central_service_catalog`(
 `productname` string,
 `scannedtime` timestamp,
 `provisionedproductname` string,
 `provisionedversion` string,
 `provisionedversionname` string ,
 `latestversionid` string ,
 `latestversionname` string ,
 `accountno` string,
 `launchedby` string ,
 `autoupdateprovisionedproduct` string,
 `productid` string ,
 `provisionedproductid` string,
 `stackname` string,
 `updatestatus` string,
 `errors` string,
 `updatetimestamp` timestamp)
ROW FORMAT SERDE
 'org.openx.data.jsonserde.JsonSerDe'
STORED AS INPUTFORMAT
 'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
 '' 
TBLPROPERTIES (
 'has_encrypted_data'='false')

To learn more about database and table creation in Amazon Athena, click here.

    1. An example that uses SQL to query the result from the table is:
SELECT * FROM <dbname>."central_service_catalog"
order by scannedtime desc, accountno, productname, provisionedproductname

See it in action

To demonstrate how the solution works after the deployment, I have manually set up a portfolio and an example of an AWS Service Catalog product called MySQL-RDS that is a MySQL 5.5.39 Amazon RDS database instance, which is available at the time of writing. In the CloudFormation template of the MySQL-RDS product, I have specified the AllowMajorVersionUpgrade properties of the AWS::RDS::DBInstance resource to be true so that the major version upgrade can be carried out.

You need to ensure that the Lambda functions in the solution have permission to use each of the underlying AWS resources of the product’s AWS CloudFormation template (Amazon RDS in this example). To do this, I applied an IAM role that has the necessary Amazon RDS permissions as the Launch Constraint of the MySQL-RDS product.

After setting up the portfolio and product, I deployed the CloudFormation template and performed the necessary configurations as outlined in Steps 2 and 3 of the previous section.

    1. After deploying the CloudFormation template, I configured the tag option for the MySQL-RDS product and the role access to the portfolio.


    1. For the purposes of this demonstration, I provisioned the product and called it MySQL. It is provisioned from MySQL-RDS product version 1.0.

    1. The database engine of the instance is MySQL 5.5.39.

    1. Next, I created a new version of the MySQL-RDS product with an updated CloudFormation template to use MySQL 5.6.39.

    1. I manually triggered the ServiceCatalogMonitor-Main Lambda function.

    1. After triggering the Lambda function, I navigated to the AWS CloudFormation console and could see that the CloudFormation stack of the provisioned product was being updated.

    1. In the AWS Step Functions console, a new execution of the ServiceCatalogMonitorStepFunction state machine was created and in the running state.

    1. You can see the details of the execution by choosing the execution name.

    1. After the Step Function execution was completed, the RDS DB instance had been upgraded to MySQL 5.6.39.


    1. The Lambda function generated a log file and stored it in the S3 bucket that was specified in the parameter of CloudFormation stack.


    1. With Amazon Athena, I can query the result from the log files in the Amazon S3 bucket with the following SQL query:
    SELECT * FROM sampledb."central_service_catalog"
order by scannedtime desc, accountno, productname, provisionedproductname

    And the result is as follows:


    I hope that this demonstration has helped you understand how the overall solution works end-to-end, and how you can easily query the results from the log files with Amazon Athena.

    Conclusion

    In this blog post, I have demonstrated how you can use CloudWatch Events rules, AWS Lambda, and AWS Step Functions to keep the provisioned products from AWS Service Catalog up to date when a new version of the product is introduced. You can selectively enable the automatic update of provisioned products using the AWS Service Catalog Tag Option. This gives you the flexibility to apply the solution in your AWS environment. You can also use Amazon Athena to easily query log files from the solution to track AWS Service Catalog provisioned products and the status of the updates.

    I hope the methods outlined here give you some ideas for automating the update of existing IT services provided to your AWS Service Catalog users. These methods give users the convenience of self-service, and at the same time, they ensure compliance to your IT standards. If you have questions about implementing the solution described in this post, start a new thread on the AWS Service Catalog Forum or contact AWS Support.

    Note: The zipped file containing the AWS Lambda code package and the CloudFormation template are hosted in an Amazon S3 bucket in US-east-1 Region. If you want to deploy the solution in other Regions, you can download the AWS Lambda code package (service-catalog-monitor.zip), host it in your S3 bucket in the Region of your choice, and update the S3Bucket properties of each Lambda function in the CloudFormation template to point to your S3 bucket before deploying the solution.

    About the Author

    Pimpuk Sansuth is a Cloud Architect with AWS Professional Services based in ASEAN. She works with AWS Enterprise customers and enjoys helping the customers in their cloud adoption journey.