Analytics
Overview
Analytics give deep insights about the traffic patterns of web applications, and help uncover opportunities to add protections using WAF, tune delivery performance using CloudFront or improve application SEO by updating its code. Edge analytics are built using server-side logs generated by CloudFront and WAF. They can be built with different AWS services or 3rd party SIEM providers according to business requirements. Client-side analytics are collected on the client side using javascript tags, independently of the application infrastructure.
Native reporting and analytics
CloudFront provides you with native reporting in the AWS console, with reports covering cache statistics (e.g. status codes, result types), most popular objects, referrers, viewer reports (e.g. devices, browsers, locations) and usage reports (e.g. transferred bytes and number of requests). When used with AWS WAF, CloudFront also exposes in the AWS console a security dashboard.
AWS WAF provides you with native dashboards that leverage CloudWatch metrics such as total requests, blocked requests, allowed requests, bots vs non bot requests, bot categories, CAPTCHA solve rate, top 10 matched rules and more, on a per-Web ACL basis. These dashboards provide enhanced visibility and help with answering questions, such as “what percent of my WAF inspected traffic is getting blocked”, “what are the top originating countries for the traffic that’s getting blocked”, “what are common attacks that WAF detects and protects me from”, “how do my traffic and traffic patterns from this week compare with last week’s”.
Client-side analytics
CloudWatch RUM provides you with analytics from your application collected on the client side, by integrating a javascript tag to your web pages. The javascript collects data from browser APIs, such as performance data (e.g. page load times and Google Core Web Vitals), and user navigation data to give you insights about the engagement of users with your website. You can analyze your application performance by filtering on specific dimensions such as the browser type, user country, or a specific page id.
Common custom analytics solutions based on CloudFront and WAF logs
Logs generated by CloudFront and WAF are required to build a custom analytics solution (i.e. personalized dashboards).
CloudFront offers two options for request logging:
- Standard logs, which can be shipped to S3, CloudWatch Logs and Kinesis Data Firehose destinations within minutes. It's configured at distribution level, and does not incur CloudFront charges, but can incur charges for the delivery, ingestion, storage or access, depending on the log delivery destination that you select.
- Real Time logs, which are delivered to Kinesis Data Stream within seconds with an additional charge of $0.01 for every 1 Million log records. It's configured at cache behaviors with possibility for sampling, and it provides more log fields.
AWS WAF offers three destinations for request logging:
- Shipping to S3, usually used for archival requirements.
- Shipping to CloudWatch logs, usually used for security analysis with CloudWatch Log insights.
- Shipping to Kinesis Firehose, usually used for security analysis with tools such as OpenSearch or 3rd party SIEMs.
To analyze AWS WAF logs, consider the following tools:
- CloudWatch Logs Insights: This feature allows you to interactively search and analyze WAF logs. It provides default queries to help identify security incidents and false positives, and you can create custom queries as needed.
- CloudWatch Contributor Insights: This feature helps create dashboards to identify top contributors to your traffic, such as top IP addresses, URIs, and user-agents, providing ongoing analysis capabilities
- Amazon Athena can be used to query WAF logs stored in Amazon S3. This service allows for complex analysis of traffic patterns, detection of false positives or negatives, and identification of new attack signatures
Dashboards using OpenSearch / Kibana
If you prefer using OpenSearch, with Kibana as a user interface for dashboarding, consider this Blog for steps to build a dashboard using CloudFront real time logs, and this blog to deploy a security dashboard for AWS WAF. Note that with OpenSearch, you can implement advanced anomaly detection based on your analytics. Learn from this blog how to use OpenSearch's anomaly detection based on WAF logs to identify abnormal behaviors, such as suspect traffic from unusual counties, and unexpected write requests for a read-heavy application.
Dashboards using Graphana
If you prefer using Graphana as a user interface for dashboarding, follow the guidance provided in this analytics solution for CloudFront based on Amazon TimeStream, and this solution for AWS WAF based on Amazon Athena.
Dashboards using CloudWatch
If you prefer using the CloudWatch ecosystem for monitoring and analytics (e.g. CloudWatch Logs Insights and CloudWatch Contributor Insights), consider this solution to deploy a custom WAF dashboard in CloudWatch.
3rd party SIEM dashboards
Third-party Security Information and Event Management (SIEM) solutions have integrations with AWS, and have off-the-shelf dashboards for CloudFront and WAF. Examples include DataDog (WAF), Sumologic ( WAF, CloudFront), and NewRelic (WAF, CloudFront).
