Monitoring Amazon Web Services (AWS) account activity can provide valuable insight into who is accessing your resources and how your resources are being used. This insight can help you make better-informed decisions that increase security and efficiency, facilitate compliance auditing, and optimize costs. Many customers choose to build custom account monitoring solutions using AWS services because these services provide an efficient way to handle a large number of activity events in real-time and the flexibility to get specific metrics.

To help you more easily monitor account activity, AWS offers the Real-Time Insights on AWS Account Activity solution, a reference implementation that automatically provisions and configures the services necessary to record and visualize resource access and usage metrics for your AWS account(s) in real-time. This solution is designed to provide a framework for visualizing access and usage metrics, allowing you to focus on adding new metrics rather than underlying infrastructure operations.  

This webpage provides best practices and guidance to consider when choosing an account-monitoring solution, as well as an overview of the Real-Time Insights on AWS Account Activity solution.

Before you implement a real-time account monitoring solution, determine the key metrics you want to track and what you want to do with those metrics. Consider solutions that provide the flexibility to easily add new metrics, are easy to deploy and use, and can scale to handle a large number of events. Keeping these general principles in mind, consider the following best practices for monitoring account activity on the AWS Cloud:

  • Choose a monitoring solution that provides a history of AWS API calls for an account, facilitates security tracking and compliance auditing, and tracks resources in real-time.
  • Consider a solution that provides highly-scalable, reliable, and low-latency data processing and storage.
  • Implement a solution that provides visibility across all operating environments. The solution should integrate with both on-premises and AWS workloads.
  • Use a solution that complements your company’s existing processes and skill sets to ensure that you can manage and modify your solution to meet future requirements. This will reduce operational complexity for developers and operators.

AWS offers a solution that uses AWS CloudTrail to log account activity, Amazon Kinesis to compute and stream metrics in real-time, and Amazon DynamoDB to durably store the computed data. Metrics are calculated for create, modify, and delete API calls for more than 60 supported AWS services. The solution also features a dashboard that visualizes your account activity in real-time. The diagram below presents the architecture you can deploy in minutes using the solution's implementation guide and accompanying AWS CloudFormation template.

  1. AWS CloudTrail logs actions taken in your AWS account, including actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services.
  2. When an action is taken, an Amazon CloudWatch event trigger sends data to a Kinesis Data Firehose delivery stream.
  3. The delivery stream archives the events in an Amazon S3 bucket and sends the data to a Kinesis Data Analytics application for processing.
  4. Once the data is processed, it is sent to Kinesis Data Streams. An AWS Lambda function reads data from the stream and sends the data in real-time to an Amazon DynamoDB table to be stored.
  5. The solution also creates an Amazon Cognito user pool, an Amazon S3 bucket, and real-time dashboard to securely read and display the account activity stored in the DynamoDB table.
Deploy Solution
Implementation Guide

What you'll accomplish:

Deploy Real-Time Insights on AWS Account Activity using AWS CloudFormation. The AWS CloudFormation template will automatically launch and configure the components necessary to monitor the resource usage of your AWS account(s) in real-time.

Log actions taken in your AWS account in real-time. Actions taken through the AWS Management Console, AWS SDKs, command line tools, and other AWS services are logged in AWS CloudTrail.

Automatically deploy a real-time dashboard that displays account activity metrics. You can customize the dashboard to include any metrics from your AWS CloudTrail logs.

What you'll need before starting:

An AWS account: You will need an AWS account to begin provisioning resources. Sign up for AWS.

Skill level: This solution is intended for IT infrastructure architects, administrators, and DevOps professionals who have practical experience architecting on the AWS Cloud.

Q: What services can this solution monitor?

This solution leverages AWS CloudTrail to record account activity. CloudTrail records account activity and service events from most AWS services. For the list of supported services, see CloudTrail Supported Services in the CloudTrail User Guide.

Q: How long after events are recorded are they available for analysis?

AWS CloudTrail monitors events that occur in your account in real-time. Some events, however, might take up to 15 minutes to arrive in Kinesis Data Firehose from CloudTrail.

Q: What metrics are displayed on the dashboard?

The solution dashboard displays a default set of metrics including the number of API calls by service, the total API calls and anomaly score, the top 10 API calls, the top 10 IAM users, the maximum calls by IP, the top calls by IP address, and the number of successful EC2 API calls.  You can also customize the dashboard to include any metrics from your AWS CloudTrail logs. For more information, see the implementation guide.

Q: Can I deploy this solution in any AWS Region?

This solution uses the Kinesis Data Firehose and Kinesis Data Analytics services, which are currently available in specific AWS Regions only. Therefore, you must deploy this solution in an AWS Region that supports these services. However, once deployed, this solution monitors all regions for events. For more information, see AWS service availability by region.

Need more resources to get started with AWS? Visit the Getting Started Resource Center to find tutorials, projects and videos to get started with AWS.

Tell us what you think