Creating a Golden AMI Pipeline Integrated with Qualys for Vulnerability Assessments

By Sagar Khasnis, Partner Solutions Architect at AWS

Last year, Amazon Web Services (AWS) released a blog post on how to create a golden Amazon Machine Images (AMI) pipeline and set up vulnerability assessments using Amazon Inspector.

Our customers loved the idea of automating the process to scan their approved AMIs in a consistent manner, and using Amazon Inspector to perform vulnerability assessments on their golden AMIs.

After several customer interactions, I realized that many of you prefer to use existing scanning tools provided by AWS Partner Network (APN) Partners such as Qualys to perform vulnerability assessments on golden AMIs. Qualys is an AWS Security Competency Partner.

In this post, I have provided an alternative to the previous sample implementation on how to create a golden AMI pipeline with Amazon Inspector. I have also provided a sample implementation for integration with a Qualys Scanner.

Qualys Virtual Scanner is available on AWS Marketplace, while the golden AMI pipeline sample configuration is available on GitHub via the Amazon Software License.

The repository contains a README guide that includes step-by-step instructions and AWS CloudFormation templates required to set up a golden AMI pipeline that allows you to create, distribute across accounts, regularly assess, and decommission golden AMIs.

Solution Overview

There are two ways to set up continuous assessments for your golden AMIs using Qualys—use the Qualys Scanner or the Qualys Cloud Agent. In this post, I will be focusing on using the Qualys Scanner for performing vulnerability assessments in your golden AMI pipeline.

Prerequisites:

1. Please read the original blog post for golden AMI pipeline creation for additional context around this implementation.
2. This post does not provide guidance on setting up a Qualys Scanner. Please refer to the guide provided by Qualys for your scanner setup, and ensure the virtual private cloud (VPC) for your golden AMI creation is accessible by your Qualys Scanner.
3. You will need to set up the Qualys Scanner to send emails to your designated admin group.
4. AWS Marketplace provides a product for installing the Qualys Scanner in your AWS environment.
5. You will need to request permission from AWS to perform penetration testing on Amazon Elastic Compute Cloud (Amazon EC2) instances.

Golden AMI Creation Process

In Figure 1, you’ll see an architecture diagram that shows how multiple event sources can trigger an SSM automation document, which in turn kicks off the golden AMI creation process. At first, an Amazon EC2 instance is launched with the AMI-ID provided, and your updates, patches, and tags are applied to it. Second, a new Amazon EC2 instance is launched and a Qualys vulnerability assessment scan is performed on it.

The results of the Qualys scan will be emailed to your admin upon configuration in the Qualys portal. Finally, this automation sends an approval notification to your admin to approve or reject the AMI as a golden AMI, based on the Qualys assessment.

Figure 1 – Multiple event sources can trigger an SSM automation document.

Step-by-Step

First, the golden AMI creation process is triggered by an SSM Automation Document (GoldenAMIAutomationDoc). You can invoke the golden AMI creation process using three event sources: Amazon CloudWatch scheduled events, manually, or using a CI/CD pipeline.

The GoldenAMIAutomationDoc performs the following steps:

• Creation of golden AMI:
  • Launch an Amazon EC2 instance with the base AMI provided as a parameter.
  • Install pre-update scripts, patches, and post-update scripts on the Amazon EC2 instance.
  • Create an AMI from the Amazon EC2 instance, which is a golden AMI candidate.
• Perform vulnerability assessment on the newly created AMI:
  • Apply tags to the candidate AMI.
  • Launch an Amazon EC2 instance with the candidate AMI.
  • Run scripts on the Amazon EC2 instance to launch a vulnerability assessment scan via API to Qualys.
  • You can set up Qualys to send you email reports after each scan.
• Create an approval request to convert the candidate AMI to golden AMI:
  • Send an approval notification to the designated IAM approver with the candidate AMI details.
  • The approver can assess the scan report from Qualys for the candidate AMI to make an approval decision.
  • Once approved, the candidate AMI ID will be stored in the SSM parameter store as a golden AMI ID.

Vulnerability Assessment Process

In Figure 2, you’ll see an architecture diagram of the golden AMI continuous vulnerability assessment process, which periodically launches instances from your approved golden AMIs, and runs a Qualys assessment scan on them to provide you continuous vulnerability assessments.

Figure 2 – Golden AMI continuous vulnerability assessment process.

Step-by-Step

The golden AMI creation process is triggered by an SSM Automation Document (RunContinuousInspection). The continuous vulnerability assessment is triggered by a CloudWatch scheduled event, which invokes an AWS Lambda function (SetupContinuousAssessmentLambdaFunction).

The SetupContinuousAssessmentLambdaFunction performs the following steps:

• Read a list of golden AMIs from the parameter store.
• Start Amazon EC2 instances for each golden AMI ID.
• Apply a set of tags to the running Amazon EC2 instances.
• Launch the SSM automation (RunContinuousInspection) for each Amazon EC2 instance.

The RunContinuousInspection performs the following steps:

• Run scripts on the Amazon EC2 instance to launch a vulnerability assessment scan via API to Qualys.
• You can set up Qualys to send you email reports after each scan.
• Stop the Amazon EC2 instance.

Deploying the Sample Golden AMI Pipeline with Qualys

This repository contains sample AWS CloudFormation templates and a README guide. Instructions on how and where to execute these templates include:

• Setting up the pipeline infrastructure in the master account. Note that if you’re using AWS Organizations this is not the master-payer account. Rather, it’s an account your Cloud Center Of Excellence (CCOE) team has identified as the master account.
• Testing the golden AMI pipeline. As part of the test, you would:
  • Create a golden AMI version (a product can have multiple golden AMI versions) you approve of. You can use your private AMI/Amazon-owned AMI/AWS Marketplace-based AMI as the source AMI.
  • Distribute the golden AMI version to one or more accounts using AWS Lambda and AWS Systems Manager.
  • Check if non-golden AMI launches are flagged as non-compliance via an AWS Config rule.
  • Launch an Amazon EC2 instance from the golden AMI in a governed manner in the child account using AWS Service Catalog.
  • Perform continuous security assessment of all active golden AMIs using Qualys Scanner.
  • Decommission a golden AMI version.

Important Notes

For details on which operating systems are supported by the golden AMI pipeline, check the support section of the README guide.

The golden AMI pipeline doesn’t give you guidance on how to harden the AMI and which agents/tools to bake into the golden AMI. The security hardening requirements and hardening-validation scripts are organization-specific. The golden AMI pipeline provides a framework for managing different aspects of the golden AMI you create and approve of.

The pipeline does not give any inputs on whether you should create environment specific or agnostic golden AMIs. How you inject the environment specific parameters to create the final AMI, is out of the context of this blog and the pipeline.

Conclusion

The golden AMI pipeline is provided to address the challenges faced by customer cloud teams. It describes a method for providing a repeatable, scalable, and approved application stack factory that increases innovation velocity, reduces effort, and increases the chief information security officer’s (CISO) confidence that IT teams are compliant in their cloud deployments.

In this post, we’ve explored how to set up a golden AMI creation pipeline with a combination of AWS services, as well as the Qualys Scanner from AWS Marketplace. This will help you automate your AMI creation process in a standardized fashion, as well as perform continuous assessments on the portfolio of hardened AMIs existing in your AWS environment.

Additionally, this solution provides you with the ability to create a compliant AMI consumption process using AWS Config and distribute your hardened AMIs across multiple regions and accounts using AWS Lambda and AWS Service Catalog.

If you have questions about implementing the solution described in this post, please contact AWS Support.

.

Qualys – APN Partner Spotlight

Qualys is an AWS Competency Partner. The Qualys Cloud Platform allows customers to deploy sensors into AWS that deliver 18 applications including Continuous Monitoring, Policy Compliance, Container Security, and more.

Contact Qualys | Solution Overview | Buy on Marketplace

*Already worked with Qualys? Rate this Partner

*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.