How to detect suspicious activity in your AWS account by using private decoy resources

As customers mature their security posture on Amazon Web Services (AWS), they are adopting multiple ways to detect suspicious behavior and notify response teams or workflows to take action. One example is using Amazon GuardDuty to monitor AWS accounts and workloads for malicious activity and deliver detailed security findings for visibility and remediation. Another tactic is to deploy decoys, also called honeypots, as an effective way to detect suspicious behavior.

In this blog post, we’ll show how you can create low-cost private decoy AWS resources in your AWS accounts and configure them to generate alerts when they are accessed. These decoy resources appear legitimate but don’t contain any useful or sensitive data and typically are not accessed in the normal course of business by your users and systems. Any attempt to access them is a clear signal of suspicious activity that should be investigated. You can use data sources like AWS CloudTrail, services like Amazon Detective, and your own security incident and event monitoring (SIEM) systems to investigate the activity further. This post is aimed at experienced AWS users and security professionals.

Detecting suspicious activity

Imagine that an unauthorized user has obtained credentials for your account. This could also be an insider, malicious or careless, using their valid credentials inappropriately. The unauthorized user might use these credentials to invoke AWS API calls to list resources in your account. As the next step, they might try to access resources that are commonly used to store sensitive data—such as objects in Amazon Simple Storage Service (Amazon S3) buckets, secrets in AWS Secrets Manager, or items in Amazon DynamoDB. They might also try to elevate their privileges by assuming other Identity and Access Management (IAM) roles in your account. In your role as a security professional, your task is to detect this suspicious behaviour and take actions in response. One approach is to learn the baseline of activities of the IAM users and roles in your account and flag any deviations from the learned baseline—this is the approach taken by GuardDuty when it generates findings such as Discovery:IAMUser/AnomalousBehavior.

This post focuses on another approach of creating private decoy resources in your account that are intended to look legitimate, but don’t have any useful or sensitive data and are not exposed publicly. These decoys are designed to alert you about suspicious activities that could indicate AWS credentials exposure or account compromise. You can use the decoys in conjunction with other techniques, such as creating deception environments and public and private honeypots to better detect suspicious activity in your accounts and applications.

The Fidelity-Isolation-Cost trilemma

In an ACM Queue article titled Lamboozling Attackers: A New Generation of Deception, Kelly Shortridge and Ryan Petrich introduced the Fidelity-Isolation-Cost (FIC) trilemma that “captures the most important dimensions of designing deception systems: fidelity, isolation, and cost.” Using their definition of the FIC trilemma, we see that decoy AWS resources can be well suited to designing deception systems:

• Fidelity – Because the decoys are actual AWS resources, they behave like other legitimate resources and have high fidelity. For example, a decoy S3 bucket behaves exactly like any other S3 bucket, with the only exception being that the object data it contains is dummy and not useful. However, the unauthorized user only discovers this fact after downloading the object data and generating an automated alert to your security team.
• Isolation – You can simply isolate the decoy AWS resources from other resources in the same account. For example, an S3 bucket is inherently isolated from other S3 buckets in the same account. An unauthorized user that can read the decoy S3 bucket does not, by doing so, get the ability to access or impact the availability of other resources in the account. The credentials obtained by the unauthorized user might have permissions to actions on other services, but the presence of the decoy S3 bucket doesn’t add to those permissions in any way.
• Cost – You can keep the cost of deception low by choosing AWS resources that have no cost or low cost to deploy, are deployed by means of automation, and require no further operation or maintenance effort. For example, an S3 bucket with several files that are a few MB in size costs a fraction of a US cent per month for storage. The API request cost should be zero, because the bucket is designed never to be accessed in the normal course of business. Choosing similar zero or low-cost resources can make it cost-effective and feasible to create such decoy resources in multiple accounts, including in Production accounts, where it’s especially important to detect suspicious activity.

Examples of private decoy AWS resources

The following table shows examples of private decoy AWS resources that are high-fidelity, high-isolation, low-cost and are suitable to be deployed in an account that has sensitive data or applications. The table also lists the CloudTrail event fields that provide the source and name for accesses to each resource. You can use these CloudTrail events to create corresponding Amazon EventBridge rules that will generate alerts and notifications.

Note: When CloudTrail Management API events are sent to EventBridge, read-only events such as Get*, List*, and Describe* are filtered out and not processed. In order to get findings for secrets and Systems Manager parameters that are being accessed, you need to alert on GetSecretValue and GetParameter API calls. Since these are not processed by EventBridge, you can instead use the fact that secrets and secure string parameters are encrypted by using AWS Key Management Service (AWS KMS), and match on the corresponding AWS KMS Decrypt API calls. This means that successful calls from an unauthorized user to GetSecretValue and GetParameter are able to be matched and alerted on.

Notifications from matching EventBridge rules can be sent to an AWS Lambda function that generates custom findings in Security Hub. These findings can then be sent to downstream systems that you might have configured in your environment, such as your SIEM system or an automated response workflow in your Security Orchestration, Automation, and Response system. Figure 1 shows this workflow.

Figure 1: Accesses to decoy resources automatically create custom Security Hub findings

Deploy the private decoy resources

We’ve provided an AWS CloudFormation template that you can use to deploy the solution. The template creates the following private decoy AWS resources in your account:

• DynamoDB table
• IAM role
• S3 bucket with a decoy S3 object
• Systems Manager SecureString parameter
• Secrets Manager secret

In addition, the CloudFormation template deploys the following resources in your account to detect accesses to the decoys and send custom findings to Security Hub:

• A CloudTrail data events trail that includes only data events from the decoy S3 bucket and DynamoDB table
• Six EventBridge rules to match specific CloudTrail API events
• Two Lambda functions with corresponding IAM roles:
  • The WriteData Lambda function is a CloudFormation custom resource that is used to create the decoy S3 object and the Systems Manager SecureString parameter
  • The Data Lambda function is a target for the EventBridge rules, and it sends custom findings to Security Hub when the decoy resources are accessed

Prerequisites

The prerequisites to deploying the solution are as follows:

• Security Hub must be enabled in the AWS Regions where the private decoys will be deployed, in order to receive custom findings.
• You must have created a CloudTrail trail to log management events for the AWS account in the Region where you deploy the private decoys. This trail can be created locally in the account or can be an organization trail. Ensure that you have enabled both read and write events, and enabled all AWS KMS events in the trail (this is the default configuration).

Deploy the solution

After you have the prerequisites set up, you can launch the CloudFormation template to deploy the private decoys.

To launch the template

1. Choose the following Launch Stack button to launch a CloudFormation stack in your account.

   

   Note: The stack will launch in the N. Virginia (us-east-1) Region. To deploy this solution into other AWS Regions, download the solution’s CloudFormation template, modify it, and deploy it to the selected Region. In order to get maximum coverage for detecting suspicious activity, we recommend that you deploy the solution into your key production accounts and Regions.

2. On the Specify stack details page, enter the stack name, then choose Next.

   The CloudFormation template will use the stack name as part of the naming of the resources that are created. We recommend that you use your organization’s existing naming conventions for stack names, and not make reference to decoy resources, because this could alert any unauthorized user to the real purpose of the resources they’re attempting to access.

   

   Figure 2: Specify stack details

3. Configure any tags or other organization-specific stack options you need, or accept the default settings, and then choose Next.
4. Review the CloudFormation settings and select the box acknowledging that AWS CloudFormation might create IAM resources with custom names, and then choose Create stack.
5. After the stack has completed deployment, the CloudFormation stack output will show the Amazon Resource Names (ARNs) of the decoy resources that were created.
   

   Figure 3: CloudFormation stack outputs

Estimated costs

This solution has been designed to keep costs as low as possible, by using services that have no associated costs (such as IAM roles or any parameters stored in Systems Manager Parameter Store), and keeping the use of paid for services (such as S3 and DynamoDB) to a minimum.

Deploying the solution as outlined in this blog post should result in a cost of less than $1 per month for a single account deployment, however please refer to the AWS Pricing Calculator where you can create a pricing estimate based on your deployment using the most up-to-date pricing information.

Test the alerts

In normal circumstances, after you configure the decoys, there will be no attempted access to these resources, and no findings will be sent to Security Hub in your account. To test that the configuration is working as expected, you can issue the following commands from a device that has programmatic access to your account where the private decoy resources have been deployed. To run each command, replace the bracketed, italicized text with your own information. You can find the details for each of the resources in the outputs section of the CloudFormation stack after it has been deployed successfully.

S3 object access

• aws s3 cp s3://<bucket_name/object_name> /tmp
• aws s3 cp s3://<bucket_name/object_name> s3://<any_existing_bucket>

IAM role assumption

• aws sts assume-role –role-arn <role_name> –role-session-name BlogTestRole

Secrets Manager access

• aws secretsmanager get-secret-value –secret-id <secret_name>

Parameter Store access

• aws ssm get-parameters –names <ssm_parameter> –with-decryption

  DynamoDB table scan

• aws dynamodb scan –table-name <table_name>

An example of what these test-generated findings looks like is shown in Figure 4.

Figure 4: Security Hub findings

Considerations

Consider the following as you deploy decoy AWS resources:

• You should consider decoy AWS resources as enhancements to your foundational security controls. Your foundational controls should include these measures:
  • Help prevent the compromise of AWS credentials and limit the privileges of credentials by implementing strong identity management and permissions management.
  • Identify and investigate alerts generated by decoy resources by implementing detective controls.
  • Implement incident response mechanisms to respond to and mitigate the potential impact of security incidents, such as a decoy AWS resource being accessed.
• You should ensure that your monitoring services and tools are configured to query the configuration of resources and not the data stored in resources. Otherwise, you might get a large volume of false positives because every time a resource is accessed, a custom finding is created in Security Hub. For example, consider a service like Security Hub Security Standards checks, or a cloud security posture management (CSPM) tool that monitors your S3 buckets by describing the properties of all buckets in your account. Such tools will find the decoy S3 bucket and will interrogate its configuration by making calls such as GetBucketPolicy and GetBucketLogging. However, as long as these tools don’t try to read data in the bucket through calls such as GetObject, the EventBridge rules that are configured as described in this post won’t generate a finding.
• As a specific example of the previous point, ensure that you don’t run a sensitive data discovery job in Amazon Macie on the decoy S3 bucket, to avoid false alerts. You can configure Amazon Macie to monitor the metadata of your S3 buckets, because those actions won’t generate alerts.
• The solution generates custom findings in Security Hub only for successful accesses of Secrets Manager secrets and Systems Manager parameters. However, both successful and unsuccessful accesses of S3 objects and DynamoDB items, and IAM role assumption, will generate custom findings in Security Hub.

Conclusion

In this post, we discussed the advantages of using private decoy AWS resources to detect suspicious activities within your account and how these decoys can complement your existing security solutions. You learned how to create private decoys, set up alerting, and ingest (and test) these alerts as custom findings into Security Hub for central visibility across your AWS environment. The solution deployment included a set of common resources as private decoys; however, the necessary code and templates can be found in our GitHub repository, and you can extend and customize these to add other resources that you want to include in your accounts.

If you would also like to learn about using CloudTrail as another method of detecting unexpected behavior within your accounts, see the blog post Using CloudTrail to identify unexpected behaviors in individual workloads for more information.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.

Want more AWS Security news? Follow us on Twitter.