Tag: DataOps

The concept of DataOps was born with the goal of solving issues prevalent in old, complex, and monolithic architectures, and to optimize data pipeline architectures. To meet the demand, Cognizant and AWS jointly built the DataHyperloop solution which provides a real-time view of DataOps and demonstrates automation of continuous integration, delivery, testing, and monitoring of data assets moving across the data lifecycle on AWS.

Enterprise data warehouses are complex and consist of database objects that need to be modified to reflect the changing needs of business, data analytics, and machine learning teams. In this post, learn about an approach to managing the evolution of enterprise-scale data warehouses based on the experience of Deloitte’s Data and AI global practice teams. The declarative tool developed by Deloitte that can automatically generate DDL statements to align Amazon Redshift’s state to an approved baseline configuration.

Amazon Managed Streaming for Apache Kafka (Amazon MSK) is a fully managed, highly available, and secure Apache Kafka service that makes it easy to build and run applications that use Kafka to process steaming data. Learn how to use the new open source Kafka Connect Connector (StreamReactor) from Lenses.io to query, transform, optimize, and archive data from Amazon MSK to Amazon S3. We’ll also demonstrate how to use Amazon Athena to query the partitioned parquet data directly from S3.