Category: Amazon Inspector

Amazon Inspector is our automated security assessment service. It analyzes the behavior of the applications that you run in AWS and helps you to identify potential security issues. In late 2015 I introduced you to Inspector and showed you how to use it (Amazon Inspector – Automated Security Assessment Service). You start by using tags to define the collection of AWS resources that make up your application (also known as the assessment target). Then you create a security assessment template and specify the set of rules that you would like to run as part of the assessment:

After you create the assessment target and the security assessment template, you can run it against the target resources with a click. The assessment makes use of an agent that runs on your Linux and Windows-based EC2 instances (read about AWS Agents to learn more). You can process the assessments manually or you can forward the findings to your existing ticketing system using AWS Lambda (read Scale Your Security Vulnerability Testing with Amazon Inspector to see how to do this).

Whether you run one instance or thousands, we recommend that you run assessments on a regular and frequent basis. You can run them on your development and integration instances as part of your DevOps pipeline; this will give you confidence that the code and the systems that you deploy to production meet the conditions specified by the rule packages that you selected when you created the security assessment template. You should also run frequent assessments against production systems in order to guard against possible configuration drift.

We have recently added some powerful new features to Amazon Inspector:

• Assessment Reports – The new assessment reports provide a comprehensive summary of the assessment, beginning with an executive summary. The reports are designed to be shared with teams and with leadership, while also serving as documentation for compliance audits.
• Proxy Support – You can now configure the agent to run within proxy environments (many of our customers have been asking for this).
• CloudWatch Metrics – Inspector now publishes metrics to Amazon CloudWatch so that you can track and observe changes over time.
• Amazon Linux 2017.03 Support – This new version of the Amazon Linux AMI is launching today and Inspector supports it now.

Assessment Reports
After an assessment runs completes, you can download a detailed assessment report in HTML or PDF form:

The report begins with a cover page and executive summary:

Then it summarizes the assessment rules and the targets that were tested:

Then it summarizes the findings for each rules package:

Because the report is intended to serve as documentation for compliance audits, it includes detailed information about each finding, along with recommendations for remediation:

The full report also indicates which rules were checked and passed for all target instances:

Proxy Support
The Inspector agent can now communicate with Inspector through an HTTPS proxy. For Linux instances, we support HTTPS Proxy, and for Windows instances, we support WinHTTP proxy. See the Amazon Inspector User Guide for instructions to configure Proxy support for the AWS Agent.

CloudWatch Metrics
Amazon Inspector now publishes metrics to Amazon CloudWatch after each run. The metrics are categorized by target and by template. An aggregate metric, which indicates how many assessment runs have been performed in the AWS account, is also available. You can find the metrics in the CloudWatch console, as usual:

Here are the metrics that are published on a per-target basis:

And here are the per-template metrics:

Amazon Linux 2017.03 Support
Many AWS customers use the Amazon Linux AMI and automatically upgrade as new versions become available. In order to provide these customers with continuous coverage from Amazon Inspector, we are now making sure that this and future versions of the AMI are supported by Amazon Inspector on launch day.

Available Now
All of these features are available now and you can start using them today!

Pricing is based on a per-agent, per-assessment basis and starts at $0.30 per assessment, declining to as low at $0.05 per assessment when you run 45,000 or more assessments per month (see the Amazon Inspector Pricing page for more information).

— Jeff;

My colleague Eric Fitzgerald wrote the guest post below in order to show you how to use an AWS Lambda function to forward Amazon Inspector findings to your ticketing and workflow systems.

— Jeff;

At AWS Re:Invent 2015 we announced Amazon Inspector, our security vulnerability assessment service that helps customers test for security vulnerabilities early and often.  Using Amazon Inspector, customers can automate security testing across development, test, and production environments, identifying security vulnerabilities as part of the entire software development, deployment, and operations lifecycle.

Customer feedback on the Amazon Inspector approach to automated security testing has been overwhelming positive.  Customers have told us that with Amazon Inspector, they are able to run security assessments more frequently and are catching security vulnerabilities earlier than they have in the past.  However, identifying the security vulnerabilities is only half the battle, the vulnerabilities that are found need to be remediated. Many of our customers have started to integrate Amazon Inspector with their workflow and ticketing systems in order to automate and accelerate the remediation workflow for Amazon Inspector findings.  We designed Amazon Inspector with this in mind and thought we would share more detail on one method for integrating Amazon Inspector findings with email, workflow, and ticketing systems.

Using AWS Lambda to push Amazon Inspector Findings to a Ticketing System
In this example, we are using an AWS Lambda function to connect Amazon Inspector to systems that can handle incident creation via email. Here’s the chain of events:

1. Amazon Inspector runs and performs a security assessment. It sends a message to an Amazon Simple Notification Service (SNS) topic at the end of the run.
2. The Lambda function is invoked by the SNS message.
3. The function fetches the findings from the security assessment.
4. The function formats and emails the findings using another SNS topic.

Along the way, the function creates the destination topic and the email subscription if necessary.

Setting up the Function
You will need to set up the function in the AWS Region where you run your Amazon Inspector assessments. If you run Amazon Inspector in more than one region, you’ll need to repeat the steps for each one. Here are the steps:

1. Create the SNS topic for Amazon Inspector.
2. Configure Amazon Inspector to send findings to the newly created topic.
3. Set up the Lambda function to fetch, format, and email the findings.

Configure an SNS Topic
The first major step is to configure an Amazon SNS topic that Amazon Inspector will notify when there are new findings, and an Amazon SNS topic that will format and send findings as email to other systems.

Navigate to the Amazon SNS Console and create a new Amazon SNS topic.  This will be the topic where Amazon Inspector will deliver notifications to.  It does not matter what you name the topic.

Next, assign the following policy to the topic.  You can do this in the Amazon SNS Console by selecting the topic, clicking on Other topic actions, and selecting Edit topic policy.  In the advanced view, replace the existing policy text with this policy:

{
  "Version": "2008-10-17",
  "Id": "inspector-sns-publish-policy",
  "Statement": [
    {
      "Sid": "inspector-sns-publish-statement",
      "Effect": "Allow",
      "Principal": {
        "Service": "inspector.amazonaws.com"
      },
      "Action": "SNS:Publish",
      "Resource": "arn:aws:sns:*"
    }
  ]
}

If you are familiar with AWS Identity and Access Management (IAM) policies, then a security best practice is to change the value of the Resource field of the policy to exactly match the Amazon SNS topic ARN, in order to restrict Amazon Inspector so that it can only publish to this topic.

Configure Amazon Inspector
Navigate to the Amazon Inspector Console, visit the Assessment templates page, and select the assessment template whose findings you want sent to the external system.  Expand the row, and you’ll see a section called SNS topics.  Click the pencil icon to the left of the Amazon SNS topics section and you’ll be able to pick the Amazon SNS topic you just created from a drop-down list.  Once you’ve selected the topic, click on Save.

Set up the Lambda Function
Navigate to the Lambda Console and create a new function using the SNS-message-python blueprint:

Select SNS for the event source and then select the SNS topic that you created in the first step:

To finish configuring the function, click Next.  Type a name and description for the function, choose the Python 2.7 runtime, and replace the sample function code with this code:

from __future__ import print_function
import boto3
import json
import datetime

sns = boto3.client('sns')
inspector = boto3.client('inspector')

# SNS topic - will be created if it does not already exist
SNS_TOPIC = "Inspector-Finding-Delivery"

# Destination email - will be subscribed to the SNS topic if not already
DEST_EMAIL_ADDR = "eric@example.com"

# quick function to handle datetime serialization problems
enco = lambda obj: (
    obj.isoformat()
    if isinstance(obj, datetime.datetime)
    or isinstance(obj, datetime.date)
    else None
)

def lambda_handler(event, context):

    # extract the message that Inspector sent via SNS
    message = event['Records'][0]['Sns']['Message']

    # get inspector notification type
    notificationType = json.loads(message)['event']

    # skip everything except report_finding notifications
    if notificationType != "FINDING_REPORTED":
        print('Skipping notification that is not a new finding: ' + notificationType)
        return 1
    
    # extract finding ARN
    findingArn = json.loads(message)['finding']

    # get finding and extract detail
    response = inspector.describe_findings(findingArns = [ findingArn ], locale='EN_US')
    print(response)
    try:
        finding = response['findings'][0]
    except OSError as err:
        print("OS error: {0}".format(err))
    except:
        print("Unexpected error:", sys.exc_info()[0])
        raise
        
    # skip uninteresting findings
    title = finding['title']
    if title == "Unsupported Operating System or Version":
        print('Skipping finding: ', title)
        return 1
        
    if title == "No potential security issues found":
        print('Skipping finding: ', title)
        return 1
    
    # get the information to send via email
    subject = title[:100] # truncate @ 100 chars, SNS subject limit
    messageBody = "Title:\n" + title + "\n\nDescription:\n" + finding['description'] + "\n\nRecommendation:\n" + finding['recommendation']
    
    # un-comment the following line to dump the entire finding as raw json
    # messageBody = json.dumps(finding, default=enco, indent=2)

    # create SNS topic if necessary
    response = sns.create_topic(Name = SNS_TOPIC)
    snsTopicArn = response['TopicArn']

    # check to see if the subscription already exists
    subscribed = False
    response = sns.list_subscriptions_by_topic( TopicArn = snsTopicArn )

    # iterate through subscriptions array in paginated list API call
    while True:
        for subscription in response['Subscriptions']:
            if ( subscription['Endpoint'] == DEST_EMAIL_ADDR ):
                subscribed = True
                break
        
        if 'NextToken' not in response:
            break
        
        response = sns.list_subscriptions_by_topic(
            TopicArn = snsTopicArn,
            NextToken = response['NextToken']
            )
        
    # create subscription if necessary
    if ( subscribed == False ):
        response = sns.subscribe(
            TopicArn = snsTopicArn,
            Protocol = 'email',
            Endpoint = DEST_EMAIL_ADDR
            )

    # publish notification to topic
    response = sns.publish(
        TopicArn = snsTopicArn,
        Message = messageBody,
        Subject = subject
        )

    return 0

Be sure to edit the DEST_EMAIL_ADDR value, and put in the actual email address that is used to send incidents to your incident management system. Optionally, you can change the name of the SNS topic that Amazon Inspector will use to send findings.

Leave the function handler (lambda_function.lambda_handler) as-is, and give the function a name:

Choose  *basic execution role from the Role drop-down. After Lambda navigates to a new page,  view the policy document, and use this one instead:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "inspector:DescribeFindings",
                "SNS:CreateTopic",
                "SNS:Subscribe",
                "SNS:ListSubscriptionsByTopic",
                "SNS:Publish"
            ],
            "Resource": "*"
        }
    ]
}

Click the Allow button to create the role and return to AWS Lambda, then leave the advanced settings as-is.

Be sure to click on Enable event source  on the Review page:

Click Create function to save the function.

And that’s it!

Ready to Roll
For any assessments where you want the findings sent to another system, just add the first Amazon SNS topic (the one you created with these instructions) to the assessment template, and ensure that new finding reports are selected for publication to that topic.

The first time you run an assessment, Amazon Inspector will notify Lambda that you have new findings, and the Lambda function that you just created will create the SNS topic (if it doesn’t already exist), subscribe the destination email address to the topic (if not already subscribed), and send the findings as email to that address.  If Lambda had to subscribe the email address to the topic, then you’ll only get one email requiring you to click a link to confirm that you want to subscribe.  After confirmation, Amazon Inspector will deliver findings to that email address.

If you want to connect to Atlassian’s Jira Service Desk, it’s super easy from here on out.  In Jira ServiceDesk, navigate to Customer Channels.  This will display the email address that can receive email and create new issues.  Put that email address into the Lambda function’s Python script and that’s where Inspector will deliver its findings.  ServiceDesk will automatically turn them into ServiceDesk issues, and you can manage your workflow there.

Stay Tuned
Thank you for using Amazon Inspector, and look for more from us soon!

— Eric Fitzgerald, Principal Security Engineer

As systems, configurations, and applications become more and more complex, detecting potential security and compliance issues can be challenging. Agile development methodologies can shorten the time between “code complete” and “code tested and deployed,” but can occasionally allow vulnerabilities to be introduced by accident and overlooked during testing. Also, many organizations do not have enough security personnel on staff to perform time-consuming manual checks on individual servers and other resources.

New Amazon Inspector
Today we are announcing a preview of the new Amazon Inspector. As the name implies, it analyzes the behavior of the applications that you run in AWS and helps you to identify potential security issues.

Inspector works on an application-by-application basis. You start by defining a collection of AWS resources that make up your application:

Then you create and run a security assessment of the application:

The EC2 instances and other AWS resources that make up your application are identified by tags. When you create the assessment, you also define a duration (15 minutes, 1 / 8 / 12 hours, or 1 day).

During the assessment, an Inspector Agent running on each of the EC2 instances that play host to the application monitors network, file system, and process activity. It also collects other information including details of communication with AWS services, use of secure channels, network traffic between instances, and so forth. This information provides Inspector with a complete picture of the application and its potential security or compliance issues.

After the data has been collected, it is correlated, analyzed, and compared to a set of built-in security rules. The rules include checks against best practices, common compliance standards, and vulnerabilities and represent the collective wisdom of the AWS security team. The members of this team are constantly on the lookout for new vulnerabilities and best practices, which they codify into new rules for Inspector.

The initial launch of Inspector will include the following sets of rules:

• Common Vulnerabilities and Exposure
• CIS Operating System Security Configuration Benchmarks
• Security Best Practices
• Runtime Behavior Analysis

Issues identified by Inspector (we call them “findings”) are gathered together and grouped by severity in a comprehensive report.

You can access the Inspector from the AWS Management Console, AWS Command Line Interface (CLI), or API.

More to Come
I plan to share more information about Inspector shortly after re:Invent wraps up and I have some time to catch my breath, so stay tuned!

— Jeff;