Game Analytics Pipeline

Please note: Amazon Kinesis Data Analytics has been renamed to Amazon Managed Service for Apache Flink.

Game Analytics Pipeline helps game developers launch a scalable serverless data pipeline to ingest, store, and analyze telemetry data generated from games and services. The Guidance supports streaming ingestion of data, allowing users to gain insights from their games and other applications within minutes. It provides a REST API and Amazon Kinesis services for ingesting and processing game telemetry. It automatically validates, transforms, and delivers data to Amazon Simple Storage Service (Amazon S3) in a format optimized for cost-effective storage and analytics. Game Analytics Pipeline provides data lake integration by organizing and structuring data in Amazon S3 and configuring AWS Glue to catalog metadata for datasets, which makes it easy to integrate and share data with other applications and users.

The Guidance is designed to provide a framework for ingesting game events into your data lake for analytics and storage, allowing you to focus on expanding functionality rather than managing the underlying infrastructure operations.

Overview

The diagram below presents the architecture you can build using the example code on GitHub.

Game Analytics Pipeline architecture

The code deploys AWS resources to enable the ingestion, analysis, monitoring, and reporting of game analytics data—setting up the infrastructure to support a serverless data pipeline. Amazon API Gateway provides REST API endpoints for registering game applications with the Guidance and for ingesting game telemetry data, which sends the events to Amazon Kinesis Data Streams. Amazon DynamoDB stores game application configurations and API keys.

Kinesis Data Streams captures streaming game data from your data producers including game clients, game servers, and other applications and enables real-time data processing by Amazon Kinesis Data Firehose and Amazon Kinesis Data Analytics. Kinesis Data Firehose consumes the streaming data from Kinesis Data Streams and invokes AWS Lambda with batches of events for serverless data processing and transformation before ingestion into Amazon Simple Storage Service (Amazon S3) for storage.

AWS Glue provides extract, transform, and load (ETL) processing workflows and metadata storage in the AWS Glue Data Catalog, which provides the basis for a data lake for integration with flexible analytics tools. Sample Amazon Athena queries analyzes game events and integration with Amazon QuickSight is available for reporting and visualization. Amazon CloudWatch monitors, logs, and generates alarms for the utilization of AWS resources and creates an operational dashboard. Amazon Simple Notification Service (Amazon SNS) provides delivery of notifications to administrators and other data consumers when CloudWatch alarms are breached.

Game Analytics Pipeline

Version 1.1.2
Last updated: 01/2022
Author: AWS

Did this Guidance help you?
Provide feedback 

Features

Launch a scalable, serverless data pipeline in AWS to analyze streaming game data

Game developers can create a scalable serverless data pipeline in AWS to ingest, store, and analyze telemetry data generated from games and services.

Quickly gain insights from games and applications

Gain insights from games and other applications within minutes from the streaming ingestion of data.

Easily integrate and share data with other applications and users

Organize and structure data in Amazon S3 to provide data lake integration and configure AWS Glue to catalog metadata for datasets.

Customize the Guidance to your game projects

Customize the code to fit your particular needs, for example, by editing the Guidance API and adapting the processing workflows and real-time streaming analytics application.
Build icon
Deploy an AWS Solution yourself

Browse our library of AWS Solutions to get answers to common architectural problems.

Learn more 
Find an APN partner
Find an AWS Partner Solution

Find AWS Partners to help you get started.

Learn more 
Explore icon
Explore Guidance

Find prescriptive architectural diagrams, sample code, and technical content for common use cases.

Learn more