Enhance real-time data analytics to get better insights on website interactions
This Guidance helps you implement server-side tagging to collect event data and perform data analysis in near real-time. You can then ingest this data to both AWS and Google Analytics™ services. Server-side tagging is a new way to use Google Tag Manager to instrument your application across devices. Server containers use the same tag, trigger, and variable model that customers have already been using, while also providing new tools that allow you to measure user activity. This Guidance deploys a Google Tag Manager server-side container to Amazon Elastic Container Service (Amazon ECS) and provides the necessary infrastructure to collect event data from a test website and emit event data to Amazon OpenSearch Service.
Please note: [Disclaimer]
Architecture Diagram
-
Part 1
-
Part 2
-
Part 1
-
Follow the steps in this architecture diagram to deploy Part 1 of this Guidance.
Step 1
The Google Tag Manager user interface (UI) acts as the control plane. The UI configures pixel tags that capture business-relevant user interactions on web properties in Server-Side Tagging mode.Step 2
As users interact with the website, the pixel code loaded in the browser from the Google Tag Manager configuration fires events.Step 3
Amazon Route 53 and Application Load Balancer send these events to a backend service.
Step 4
Google’s Server-Side Tag Manager Service is deployed on Amazon Elastic Container Service (Amazon ECS). The Server-Side Tagging Service acts as the data collector and receives events from Application Load Balancer.
Step 5
The Server-Side Tagging Preview Service deployed on Amazon ECS tests and previews the tag configuration. This service is connected to the client side and the Server-Side Tagging Service through an Application Load Balancer and Route 53.
Step 6
The Server-Side Tagging Service sends HTTPS Post requests to an internal, private Amazon API Gateway endpoint. This endpoint is connected to the Amazon Virtual Private Cloud (Amazon VPC) with an interface endpoint.
Step 7
These events are also sent to the Google Analytics™ service and any third-party data collectors that are already in use, requiring no changes to the front-end code. -
Part 2
-
Follow the steps in this architecture diagram to deploy Part 2 of this Guidance.
Step 8
Amazon Kinesis Data Steams receives requests from API Gateway through a direct integration.Step 9
Amazon Kinesis Data Firehose receives data from Kinesis Data Streams and stores it in an Amazon Simple Storage Service (Amazon S3) bucket for long-term storage and historical or ad-hoc analysis.Step 10
Optionally, use AWS Lambda with Kinesis Data Firehose to enrich events or remove sensitive data from the events before storing them in Amazon S3.
Step 11
AWS Glue Crawler and AWS Glue Data Catalog capture the metadata, enabling Amazon Athena to query the data.
Step 12
Amazon QuickSight builds interactive dashboards and uses Athena to query data.
Step 13
Lambda is configured to act as a secondary receiver of the same Kinesis Data Streams event stream.
Step 14
The Lambda function, integrated with Kinesis Data Streams, loads data into Amazon OpenSearch Service for near real-time analysis and observability.Step 15
Website event data stored in Amazon S3 serves as a foundation for building a customer 360 profile, web personalization using Amazon Personalize, privacy-enhanced data collaboration with advertising partners using AWS Clean Rooms, and artificial intelligence and machine learning (AI/ML) use cases.
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational Excellence
This Guidance provides built-in observability. Each service in this Guidance publishes metrics to Amazon CloudWatch, through which you can configure dashboards and alarms. You can then use alarms or Amazon Simple Notification Service (Amazon SNS) to notify incident management systems of events and escalate based on event severity.
-
Security
The data at rest in the S3 bucket is encrypted using AWS Key Management Service (AWS KMS) keys. The data in transit is encrypted and transferred over HTTPS. The Amazon ECS clusters run in an Amazon VPC. The connectivity between Amazon ECS clusters also run in Amazon VPC, and the connectivity between Amazon ECS and API Gateway is routed through a secure Amazon VPC interface endpoint.
-
Reliability
You can use OpenSearch Service-automated snapshots for backup and recovery. You can back up Kinesis Data Firehose data to Amazon S3, and static content is stored in Amazon S3, which offers industry-leading storage durability. Amazon ECS clusters come with capabilities to stop non-responding containers and create new ones that handle incoming traffic without manual interventions.
-
Performance Efficiency
All components of this Guidance are co-located in a single AWS Region and multiple Availability Zones. This Guidance also uses a serverless stack so that you don’t have to make infrastructure decisions about locations, aside from your choice of Region and Availability Zone. You can use automated deployments to deploy the Guidance components into any Region quickly, providing you with reduced latency and support for data residency requirements.
-
Cost Optimization
This Guidance uses a multi-Availability Zone deployment to support the high availability and resilience requirements for server-side tagging and the near real-time event collection workload. There will be charges for data transfer within the Region and charges for data transfer out to Google Analytics™ service and each third-party collector service. As an industry standard, we recommend estimating your data transfer out charges early on in the deployment.
-
Sustainability
All of the services used in this architecture are managed services. With Amazon ECS, you can horizontally scale and implement an elastic scaling mechanism. Additionally, by decoupling microservices within the architecture, you can scale services in a way that allocates instance types and number of instances based on the exact amount of resources used by your workloads.
Implementation Resources
A detailed guide is provided to experiment and use within your AWS account. Each stage of building the Guidance, including deployment, usage, and cleanup, is examined to prepare it for deployment.
The sample code is a starting point. It is industry validated, prescriptive but not definitive, and a peek under the hood to help you begin.
Related Content
Guidance for Near Real-Time Personalized Recommendations on AWS
Clickstream Analytics on AWS
This solution collects, ingests, analyzes, and visualizes clickstream data from your websites and mobile applications
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.