AWS Big Data Blog

Amazon Redshift system table integration with Amazon S3 Tables automatically delivers your system table logs to Amazon S3 Tables in Apache Iceberg format. You can retain this data well beyond the 7-day limit for compliance, auditing, and cross-warehouse observability, without custom ETL pipelines or cluster resource consumption.

Learn how to track Amazon SageMaker Unified Studio project costs by custom tags. This serverless solution enriches AWS Cost and Usage Report (CUR) data with custom project tags and visualizes cost by CostCenter, Team, or Environment in an Amazon Quick Sight dashboard.

Learn how to secure Amazon SageMaker Unified Studio by integrating it with an external SAML identity provider such as Okta. This post shows you how to apply conditional access policies that enforce device compliance, IP-based restrictions, and multi-factor authentication for your data and AI workloads.

Learn how to run fast analytical queries directly against raw log and trace data in Amazon OpenSearch Service using PPL and SQL. Follow a single incident investigation, one query at a time, and see how the new optimized engine answers each question directly from raw spans.

Leading everyday app across the Middle East and North Africa, talabat, built a hybrid multi-cloud lakehouse that keeps a single Apache Iceberg copy of streaming data on Amazon S3 Tables while letting Google BigQuery query it in place, eliminating cross-cloud data duplication and schema-synchronization overhead.

Agentic AI applications now observe, reason, and act on streaming data in production. This post presents three architecture patterns that form a unified streaming backbone for the agentic AI era: streaming feature engineering with real-time inference, event-driven agent invocation, and real-time context synchronization.

With Amazon MSK, you can now configure custom domain names for provisioned clusters using a single configuration property that works identically on ZooKeeper and KRaft. Define the domain once and Amazon MSK applies it across every broker, so custom domain names keep working as the cluster scales.

Learn how Zepto, India’s fast-growing quick-commerce platform, migrated Amazon OpenSearch Service to OpenSearch Optimized (OR2) instances to scale sub-second product search across hundreds of delivery hubs, achieving over 100% higher indexing throughput and 30% cost savings while serving the same workload on two-thirds the data nodes.

IAM authentication with OAuth 2.0 lets clients connect to Amazon MQ for RabbitMQ using their existing IAM identity instead of static broker-local credentials. This post covers the key rabbitmq.conf configuration for using AWS IAM as an OAuth 2.0 provider and shows a multi-tenant example with vhost isolation enforced by IAM roles and broker scope aliases.

Amazon MQ for RabbitMQ supports OAuth 2.0, LDAP, and HTTP-based authentication backends so you can connect your broker to the identity infrastructure you already use. This post explains how each approach works and helps you decide which one fits your use case.