Overview
Automated Data Analytics on AWS enables you to derive meaningful insights from data in a matter of minutes through a simple and intuitive user interface.
This solution helps to easily consolidate data distributed across siloes, apply fine-grained governance controls, and query data through a tailored user experience that is abstracted from underlying AWS services. The solution is quickly deployed to an AWS account through a single click, removing the need for deep technical expertise.
What's new | April 2023
- Added connector support to ingest data from DynamoDB, MongoDB and CloudTrail.
- Upgraded the AWS Glue library to v3.0.
To find out about other new features, refer to the Revisions page.
Benefits
Ready to use out of the box, no need to build your own platform. If you are early in your data journey, get started quickly with the solution, and expand your capabilities over time.
Removes the heavy lifting of joining and managing disparate datasets.
Anyone with SQL skills can quickly and easily derive insights from their data.
Users can share datasets and queries across teams with flexible controls to enforce access protection.
Technical details
The diagram below presents the architecture that will be automatically deployed following the steps in the solution's Implementation Guide and accompanying AWS CloudFormation template.
Once deployed, users will access the application through a standalone user interface that abstracts away the underlying AWS services used.
Learn how to use this AWS Solution, ingest data from a range of data sources and more in the implementation guide.
Step 1
The AWS CloudFormation template provisions the following infrastructure and services provided by the solution.
Step 2
Amazon CloudFront and AWS WAF for static website hosting distribution and protection. An Amazon DynamoDB table is used to manage and provide persistent notifications in the user interface.
Step 3
Federated Identity: An Amazon Cognito user pool manages federating and storing users from external identity providers (IDPs).
Step 4
An Amazon DynamoDB table to store group policy statement, and an Amazon Cognito user pool for managing federated user authentication.
AWS Identity and Access Management and Amazon API Gateway to manage permissions and proxying egress requests from external clients.
Step 5
Amazon EventBridge for event-driven application messaging between micro-services and notifications for user.
Step 6
- AWS Lambda functions for handling API requests (NodeJS & Java), and deploy dynamic infrastructure for each data product (NodeJS).
- AWS Step Functions for managing the lifecycle of data products, and asynchronous life-cycle of query execution.
- Amazon S3 buckets for storing processed data, user-defined scripts, and file uploads.
- AWS Glue tables and resources for handling the data extract, transform, and load (ETL) processing.
- Amazon Athena for performing federated queries which stores results in Amazon S3 buckets.
- Amazon DynamoDB data stores for saved queries, query history, and query caching, and governance metadata.
Step 7
- AWS Lambda functions for handling source import.
- AWS CloudFormation stack to manage resources.
- AWS Step Function for orchestrating lifecycle management.
- AWS Glue crawlers, data catalogues, and jobs for ETL.
- AWS Secrets Manager to store external credentials.
- Amazon ECS tasks for processing large data ingestion jobs.
- Amazon Athena and Amazon Comprehend for detecting PII entities.
Step 8
Ingress (Data Connectors): Automated Data Analytics on AWS supports multiple source data connectors out-of-the-box including file upload, AWS S3, Amazon Kinesis, Amazon CloudWatch, Google BigQuery, Google Cloud Storage, and Google Analytics, and databases such as MySql5, Postgres, Oracle, and MS SQL.
Step 9
Egress (Clients): Automated Data Analytics on AWS support both JDBC and ODBC standards for consuming data from common clients.
Step 1
The AWS CloudFormation template provisions the following infrastructure and services provided by the solution.
Step 2
Amazon CloudFront and AWS WAF for static website hosting distribution and protection. An Amazon DynamoDB table is used to manage and provide persistent notifications in the user interface.
Step 3
Federated Identity: An Amazon Cognito user pool manages federating and storing users from external identity providers (IDPs).
Step 4
An Amazon DynamoDB table to store group policy statement, and an Amazon Cognito user pool for managing federated user authentication.
AWS Identity and Access Management and Amazon API Gateway to manage permissions and proxying egress requests from external clients.
Step 5
Amazon EventBridge for event-driven application messaging between micro-services and notifications for user.
Step 6
- AWS Lambda functions for handling API requests (NodeJS & Java), and deploy dynamic infrastructure for each data product (NodeJS).
- AWS Step Functions for managing the lifecycle of data products, and asynchronous life-cycle of query execution.
- Amazon S3 buckets for storing processed data, user-defined scripts, and file uploads.
- AWS Glue tables and resources for handling the data extract, transform, and load (ETL) processing.
- Amazon Athena for performing federated queries which stores results in Amazon S3 buckets.
- Amazon DynamoDB data stores for saved queries, query history, and query caching, and governance metadata.
Step 7
- AWS Lambda functions for handling source import.
- AWS CloudFormation stack to manage resources.
- AWS Step Function for orchestrating lifecycle management.
- AWS Glue crawlers, data catalogues, and jobs for ETL.
- AWS Secrets Manager to store external credentials.
- Amazon ECS tasks for processing large data ingestion jobs.
- Amazon Athena and Amazon Comprehend for detecting PII entities.
Step 8
Ingress (Data Connectors): Automated Data Analytics on AWS supports multiple source data connectors out-of-the-box including file upload, AWS S3, Amazon Kinesis, Amazon CloudWatch, Google BigQuery, Google Cloud Storage, and Google Analytics, and databases such as MySql5, Postgres, Oracle, and MS SQL.
Step 9
Egress (Clients): Automated Data Analytics on AWS support both JDBC and ODBC standards for consuming data from common clients.
Note: Before you launch the solution in the AWS Management Console, ensure that you meet the prerequisites in the implementation guide.
Related content
Learn how Stax, a global B2B software-as-a-service company, enhanced its analytics using Automated Data Analytics (ADA) on AWS.