Real-Time Web Analytics with Kinesis Data Analytics

What does this AWS Solutions Implementation do?

The Real-Time Web Analytics with Kinesis Data Analytics solution automatically provisions the services necessary to track and visualize website clickstream data in real-time. This solution is designed to provide a framework for analyzing and visualizing metrics, allowing you to focus on adding new metrics rather than managing the underlying infrastructure.

Version 1.1.2 of the solution uses the most up-to-date Node.js runtime. Version 1.0.2 uses the Node.js 8.10 runtime, which reaches end-of-life on December 31, 2019. To upgrade to version 1.1.2, you must deploy the solution as a new stack. For more information, see the deployment guide.

AWS Solutions Implementation overview

AWS offers a solution that uses beacon web servers to log requests from a user’s web browser, Amazon Kinesis Data Firehose to capture website clickstream data, Amazon Kinesis Data Analytics to compute metrics in real-time, and Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB to durably store metric data. The solution also features a dashboard that visualizes your account activity in real-time. The diagram below presents the architecture you can deploy in minutes using the solution's implementation guide and accompanying AWS CloudFormation template.

Click to enlarge

Real-Time Web Analytics with Kinesis Data Analytics architecture

Web beacon servers log requests from a user’s web browser and send the data to a Kinesis Data Firehose delivery stream. The delivery stream archives the events in an Amazon S3 bucket and sends the data to a Kinesis Data Analytics application for processing.

Once the data is processed, it is sent to Kinesis Data Streams. An AWS Lambda function reads data from the stream and sends the data in real-time to an Amazon DynamoDB table to be stored.

The solution also creates an Amazon Cognito user pool, an Amazon S3 bucket, an Amazon CloudFront distribution, and real-time dashboard to securely read and display the account activity stored in the DynamoDB table.

Show less

Real-Time Web Analytics with Kinesis Data Analytics

Version 1.1.2
Last updated: 12/2019
Author: AWS

Estimated deployment time: 10 min

Source code

CloudFormation template

View deployment guide

Launch in the AWS Console

Deploy with an AWS IQ expert

Deployment resources

Download deployment guide

AWS Solutions Implementation resources »
Contact us »

Use the button below to subscribe to solution updates.

Subscribe to RSS feed

Note: To subscribe to RSS updates, you must have an RSS plug-in enabled for the browser you are using.

Rate this Solution

Features

Real-time visualizations

Real-Time Web Analytics with Kinesis Data Analytics enables you to track website activity in real-time. Visualize web usage metrics including events per hour, visitor count, user agents, abnormal events, aggregate event count, referrers, and recent events. You can also customize this solution to include additional metrics.

Beacon server performance metrics

This solution provides an optional dashboard you can use to monitor the performance of your beacon web servers with Amazon CloudWatch. The dashboard displays custom operational metrics for the servers, including the number of healthy servers, the average processed network packets, aggregate requests, 5XX errors, and Amazon DynamoDB throughput capacity and throttling.

Real-time dashboard

Automatically deploy a real-time dashboard that displays web usage metrics. You can customize the dashboard to include any metrics.

Deploy a Solution yourself

Browse our library of AWS Solutions Implementations to get answers to common architectural problems.

Learn more

Find an APN Partner

Find AWS certified consulting and technology partners to help you get started.

Learn more

Explore Solutions Consulting Offers

Browse our portfolio of Consulting Offers to get AWS-vetted help with solution deployment.

Learn more