Centralized Logging

The Centralized Logging solution contains the following components: log ingestion, log indexing, and visualization. You must deploy the AWS CloudFormation template in the AWS account where you intend to store your log data.

1. Log ingestion: Amazon CloudWatch Logs destinations deploy in the primary account and are created with the required permissions in each of the selected Regions. CloudWatch Logs subscription filters can be configured for log groups to be streamed to the Centralized Logging account. An optional demo AWS CloudFormation template can be deployed to generate sample CloudWatch Logs for AWS CloudTrail, Amazon Virtual Private Cloud (Amazon VPC) flow logs, and an Amazon Elastic Compute Cloud (Amazon EC2) web server.
   
2. Log indexing: A centralized Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose are provisioned to index log events on the centralized Amazon Elasticsearch Service (Amazon ES) domain. The CloudWatch Logs destinations created to stream log events, have Kinesis Data Streams as their target. Once the log events stream to Kinesis Data Streams, the service invokes an AWS Lambda function to transform each log event to an Amazon ES document, which is then put into Kinesis Data Firehose. You can monitor Kinesis Data Firehose while it sends custom CloudWatch Logs containing detailed monitoring data for each delivery stream.
   
3. Visualization: Amazon ES and Kibana provide data visualization and exploration support. An Amazon ES domain is created inside an Amazon VPC, preventing public access to the Kibana dashboard. Optionally, a Microsoft Windows Jumpbox Server can be launched to access the Amazon ES cluster and Kibana dashboard.