AWS Big Data Blog

Category: Analytics

Optimizing Apache HBase availability on Amazon EMR, Part 2: Diagnose and tune compaction

Optimizing Apache HBase availability on Amazon EMR, Part 2: Diagnose and tune compaction

Compaction keeps HBase read performance healthy, but when it runs during peak hours it drives latency spikes, write stalls, and RegionTooBusyException errors. In Part 2 of this series, you learn how to confirm compaction is the cause on Amazon EMR, tune the right parameters in the least-invasive order, and know when to stop and contact AWS Support.

Build a multi-team AI and analytics environment on Amazon SageMaker Unified Studio with Terraform

Build a multi-team AI and analytics environment on Amazon SageMaker Unified Studio with Terraform

Learn how to use the Amazon SageMaker Unified Studio Terraform module to provision a multi-team AI and analytics platform (domains, blueprints, project profiles, and projects) for data analytics, machine learning, and generative AI teams in a single terraform apply, with shared governance and least-privilege security defined entirely as code.

How Samsung Ads federated Snowflake Iceberg tables using Amazon Cognito

How Samsung Ads federated Snowflake Iceberg tables using Amazon Cognito

Learn how Samsung Ads federates Snowflake-managed Apache Iceberg tables into AWS Lake Formation and queries them from Amazon Athena using Amazon Cognito for OAuth2 machine-to-machine authentication, with no second copy of the data to maintain.

Allocate shared Amazon MSK costs to Kafka topics for cost visibility

Allocate shared Amazon MSK costs to Kafka topics for cost visibility

Amazon MSK bills at the cluster level, with no per-topic breakdown. This post shows how to allocate shared MSK cluster costs to individual Kafka topics by combining AWS Cost and Usage Report billing with per-topic CloudWatch or Prometheus metrics, giving each team a usage-based view of its spend.

Governance compliance beyond the catalog in Amazon SageMaker

Governance compliance beyond the catalog in Amazon SageMaker Unified Studio

Amazon SageMaker provides rich governance signals such as metadata forms, data quality scores, glossary coverage, and subscription data, but they live in separate APIs. This post shows how to join them into a single enriched Iceberg table and build a compliance scorecard you can query, score, and adapt to your governance policy.

How Intuit migrated from Apache Hive Metastore to AWS Glue Data Catalog

How Intuit migrated from Apache Hive Metastore to AWS Glue Data Catalog

How Intuit migrated its data catalog from Apache Hive Metastore to AWS Glue Data Catalog on its journey to an open, secure lakehouse. This post covers the challenges, the phased bidirectional-sync solution, and the benefits: reduced operational overhead, improved data discovery, and tighter integration with AWS analytics services.

46% fewer nodes, 2× faster indexing: Confluent on Amazon OpenSearch Service

46% fewer nodes, 2× faster indexing: Confluent on Amazon OpenSearch Service

How Confluent modernized its Amazon OpenSearch Service observability hot tier, ingesting hundreds of terabytes of logs per day, by migrating from r6gd to the purpose-built oi2 instance family. A measure-first approach cut the fleet 46%, roughly halved indexing latency, and cleared write backpressure, lowering cost even at a higher per-node price.