AWS Cloud Operations Blog

How to self-service manage AWS Auto Scaling groups and Amazon Redshift with AWS Service Catalog Service Actions

Some of the customers I work with provide AWS Service Catalog products to their end-users to enable self-service for launching and managing Amazon Redshift, EMR clusters or web applications at scale using AWS Auto Scaling groups. These end-users would like the ability to self-manage these resources, for example, be able to take a snapshot of an instance or data warehouse.  With AWS Service Catalog, end-users can launch data warehouse products using Redshift, a web farm using EC2 or a Hadoop instance using EMR.

In this blog post, I will show you how to enable your end-users by creating self-service actions using AWS Service Catalog Service Actions with AWS Systems Manager. You will also learn how to use the Service Actions feature to manage these products, for example, how to start or stop EC2 instances running under an auto scaling group and also how to backup EC2 and Redshift.

This solution uses the following AWS services. Most of the resources are set up for you with an AWS CloudFormation stack:

Background

Here are some of AWS Service Catalog concepts referenced in this post. For more information, see Overview of AWS Service Catalog.

  • A product is a blueprint for building the AWS resources to make available for deployment on AWS, along with the configuration information. Create a product by importing an AWS CloudFormation template, or, in case of AWS Marketplace-based products, by copying the product to AWS Service Catalog. A product can belong to multiple portfolios.
  • A portfolio is a collection of products, together with the configuration information. Use portfolios to manage user access to specific products. You can grant portfolio access for an AWS Identity and Access Management (IAM) user, IAM group, or IAM role level.
  • A provisioned product is an AWS CloudFormation stack; that is, the AWS resources that are created. When an end-user launches a product, AWS Service Catalog provisions the product from an AWS CloudFormation stack.
  • Constraints control the way users can deploy a product. With launch constraints, you can specify a role that the AWS Service Catalog can assume to launch a product.

Solution overview­

The following diagram maps out the solution architecture.

 

Here’s the process for the administrator:

  1. The administrator creates an AWS CloudFormation template for an auto scaling group.
  2. The administrator then creates an AWS Service Catalog product based on the CloudFormation template
  3. AWS Systems Manager is then used to create a SSM automation document that will manage the EC2 instances under an auto scaling group and Redshift cluster. An AWS Service Catalog self service action is then created based on the automation documents and attached to the AWS Service Catalog auto scaling group and Redshift product.

Here’s the process when the end-user launches the auto scaling group product:

  1. The end-user selects and launches an AWS Service Catalog auto scaling group or the Redshift product.
  2. The end-user uses the AWS Service Catalog console to select the auto scaling group, or the Redshift product then chooses the self-service action to stop or start the EC2 instances or create a snapshot of Redshift.
  3. Behind the scene, invisible to the end-user the SSM automation document stops or starts the EC2 instances or takes a snapshot of Redshift.

Step 1: Configuring an environment

To get the setup material:

  1. Download the sc_ssm_autoscale.zip file with the configuration content.
  2. Unzip the contents and save them to a folder. Note the folder’s location.

Create your AWS Service Catalog auto scaling group and Redshift products:

  1. Log in to your AWS account as an administrator. Ensure that you have an AdministratorAccess IAM policy attached to your login because you’re going to create AWS resources.
  2. In the Amazon S3 console, create a bucket. Leave the default values except as noted.
    •  Bucket name – scssmblog-<accountNumber>. (No dashes in the account number e.g. scssmblog-999999902040)

To upload content to the new bucket:

  1. Select your bucket, and choose Upload, Add files.
  2. Navigate to the folder that contains the configuration content. Select all the files and choose Open. Leave the default values except as noted.
  3. After the Review page, from the list of files, select the sc_setup_ssm_autoscale.json file.
  4. Right-click the link under Object URL and choose Copy link address.

To launch the configuration stack:

  1. In the AWS CloudFormation console, choose Create Stack, Amazon S3 URL, paste the URL you just copied, and then choose Next.
  2. On the Specify stack details page, specify the following:
    1. Stack name: scssmblogSetup
    2. S3Bucket: scssmblog-<accountNumber>
    3. SCEndUser: The current user name
  3. Leave the default values except as noted.
  4. On the Review page, check the box next to I acknowledge that AWS CloudFormation might create IAM resources with custom names, and choose Create.
  5. After the status of the stack changes to CREATE COMPLETE, select the stack and choose Outputs to see the output.

Find the ServiceCatalog entry choose the URL to the right.

Congratulations! You have completed the setup.

Step 2: Creating the AWS SSM automation document

You will repeat these steps for the Redshift snapshot document.

  1. Open the file ssmasg_stop.json for ASG action redshift_snapshot.json for the Redshift document next you downloaded in the previous step.
  2. Copy the contents.
  3. Log into the AWS Systems Manager console as an admin user.
  4. Choose Documents from the menu at the bottom left.
  5. Choose Create document:
    • Name – SCAutoScalingEC2stop for ASG – SCSnapshotstop for Redshift
    • Target Type –

      • /AWS::AutoScaling::AutoScalingGroup for ASG
      • /AWS::Redshift::Cluster   for Redshift
    • Document type – Automation document
    • JSON – paste the content you copied from step 2
    • Choose Create document

You will see a green banner saying your document was successfully created.

 

Step 3: Create a AWS Service Catalog self-service action

  1. Log into the AWS Service Catalog console as an admin user.
  2. On the left navigation pane, choose Service actions.
  3. Choose Create new action.
  4. On the Define page choose Custom documents.
  5. Choose the document you just created for ASG.
  6. Choose Next.
  7. On the Configure page, leave the default values.
  8. Choose Create action.

You will see a banner saying the product has been created and is now ready to use.
Repeat for the Redshift product.

Step 4: Associate action to the product

  1. On the Service actions page, choose the action you created.
  2. Choose Associate action.
  3. Choose the AutoScaling product.
  4. Choose the Version.
  5. Choose Associate action.

Repeat for the Redshift product.

Congratulation! your new service action has been associated with the product. The next step is to deploy. the AutoScaling and Redshift products and use the new self-service action.

Step 5: Launching the AWS Service Catalog product

Redshift

  1. Log into the AWS Service Catalog console as an admin or end-user.
  2. On the left navigation pane on top, choose Products list.
  3. Choose the Redshift product.
  4. Choose LAUNCH PRODUCT.
  5. Enter a name – myredshift
  6. Choose Next.
  7. On the Parameters page:
    • DBName – mydb001
    • MasterUserPassword – enter a password
  8. Choose Next.
  9. On the TagOptions page choose Next.
  10. On the Notifications page choose Next.
  11. On the Review page choose Launch.

Auto Scaling Group

  1. Log into the AWS Service Catalog console as an admin or end-user.
  2. On the left navigation pane on top, choose Products list.
  3. Choose the AutoScaling product.
  4. Choose LAUNCH PRODUCT.
  5. Enter a name – myscacg
  6. Choose Next.
  7. On the Parameters page:
    • Serverpostfix – default
    • Imageid – enter an amz-linux ami for your current region
  8. Choose Next.
  9. On the TagOptions page choose Next.
  10. On the Notifications page choose Next.
  11. On the Review page choose Launch.

Wait for the status to change to Completed.

 

Step 6: Executing the self-service action

Auto Scaling Group

  1. Choose Actions.
  2. Choose the self-service action you created SCAutoScalingEC2stop.
  3. Choose RUN ACTION to confirm.

Redshift

  1. Choose Actions.
  2. Choose the self-service action you created SCSnapshotstop.
  3. Choose RUN ACTION to confirm.

Congratulations, you have successfully executed the new self-service action.

 

Cleanup process

To avoid incurring cost, please delete resources that are not needed. You can terminate the Service Catalog product deployed by selecting Action then Terminate.

 

Conclusion
In this post, you learned an easy way to backup Redshift databases and to manage EC2 instances in an auto scaling group. You also saw how there’s an extra layer of governance and control when you use AWS Service Catalog to deploy resources to support business objectives.

About the Author

Kenneth Walsh is a New York-based Solutions Architect focusing on AWS Marketplace. Kenneth is passionate about cloud computing and loves being a trusted advisor for his customers. When he’s not working with customers on their journey to the cloud, he enjoys cooking, audio books, movies, and spending time with his family and dog.