Real-Time Web Analytics with Kinesis Data Analytics

Please note: Amazon Kinesis Data Analytics has been renamed to Amazon Managed Service for Apache Flink.

Real-Time Web Analytics with Kinesis Data Analytics automatically provisions the services necessary to track and visualize website clickstream data in real-time. It is designed to provide a framework for analyzing and visualizing metrics, allowing you to focus on adding new metrics rather than managing the underlying infrastructure.

Overview

This Guidance uses beacon web servers to log requests from a user’s web browser, Amazon Kinesis Data Firehose to capture website clickstream data, Amazon Kinesis Data Analytics to compute metrics in real-time, and Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB to durably store metric data. It also features a dashboard that visualizes your account activity in real-time. The diagram below presents the architecture you can build using the example code on GitHub.

Real-Time Web Analytics with Kinesis Data Analytics architecture

Web beacon servers log requests from a user’s web browser and send the data to a Kinesis Data Firehose delivery stream. The delivery stream archives the events in an Amazon S3 bucket and sends the data to a Kinesis Data Analytics application for processing.

Once the data is processed, it is sent to Kinesis Data Streams. An AWS Lambda function reads data from the stream and sends the data in real-time to an Amazon DynamoDB table to be stored.

The Guidance also creates an Amazon Cognito user pool, an Amazon S3 bucket, an Amazon CloudFront distribution, and real-time dashboard to securely read and display the account activity stored in the DynamoDB table.

Real-Time Web Analytics with Kinesis Data Analytics

Version 1.1.2
Last updated: 12/2019
Author: AWS

Additional resources

Did this Guidance help you?
Provide feedback 

Features

Real-time visualizations

Real-Time Web Analytics with Kinesis Data Analytics enables you to track website activity in real-time. Visualize web usage metrics including events per hour, visitor count, user agents, abnormal events, aggregate event count, referrers, and recent events. Customize this Guidance to include additional metrics.

Real-time dashboard

Automatically deploy a real-time dashboard that displays web usage metrics. You can customize the dashboard to include any metrics.

Beacon server performance metrics

Use the optional dashboard to monitor the performance of your beacon web servers with Amazon CloudWatch. The dashboard displays custom operational metrics for the servers, including the number of healthy servers, the average processed network packets, aggregate requests, 5XX errors, and Amazon DynamoDB throughput capacity and throttling.
Build icon
Deploy an AWS Solution yourself

Browse our library of AWS Solutions to get answers to common architectural problems.

Learn more 
Find an APN partner
Find an AWS Partner Solution

Find AWS Partners to help you get started.

Learn more 
Explore icon
Explore Guidance

Find prescriptive architectural diagrams, sample code, and technical content for common use cases.

Learn more