Run Spark 31% faster and optimize compute costs with Amazon S3 Express One Zone on Amazon EMR

As your Spark datasets grow, storage latency often becomes the constraint, impeding application performance. Query runtimes stretch, and the bottleneck shifts from compute to how fast each node can read from Amazon Simple Storage Service (Amazon S3).

We benchmarked this directly on Amazon EMR with TPC-DS at 3 TB scale. On an 8-node Graviton4 cluster running Amazon EMR 7.13, Amazon S3 Express One Zone reduced total benchmark runtime by 31 percent. Of the 105 queries, 103 ran faster with no application code changes. At 10 TB, the gap widens to 47 percent.

In this post, we cover the methodology, show why the improvement grows with data volume, and provide what you need to replicate the comparison on your own workloads.

S3 Express One Zone is purpose-built for low-latency access. It streamlines the request authorization mechanism to reduce per-request overhead and co-locates storage with compute in a single Availability Zone to avoid network hops. For I/O-bound analytics workloads reading millions of objects, these performance benefits compound across Spark tasks.

Performance advantage

The performance advantage requires no changes to your SQL queries or Spark application code. Point your workload at a directory bucket that stores objects in the S3 Express One Zone storage class, and the EMR S3A connector automatically optimizes its read strategy. To enable this, follow these two setup considerations:

• Launch your EMR cluster in the same Availability Zone as your directory bucket. S3 Express One Zone stores data in a single Availability Zone. This co-location avoids cross-AZ hops and delivers the lowest read latency.
• Set spark.sql.sources.fastS3PartitionDiscovery.enabled=false. The default fast partition discovery relies on an S3 List API parameter that directory buckets do not support. Without this setting, partition discovery fails.

Prerequisites

To reproduce this benchmark, you need the following:

• An AWS account with permissions to create EMR clusters and S3 buckets
• A directory bucket in your target Availability Zone
• An EMR 7.13 cluster launched in the same Availability Zone as the directory bucket
• The TPC-DS data generator (or pre-generated Parquet data at 3 TB scale factor)
• Familiarity with Apache Spark SQL on Amazon EMR

Run benchmark

We ran the TPC-DS benchmark at 3 TB scale factor: 924 GB of Snappy-compressed Parquet data across 24 tables and 5,484 objects. All 105 queries ran three times, and we averaged the per-query runtimes. Every query passed on every run (105/105 across three rounds).

To get started:

1. Create a directory bucket in your target Availability Zone.
2. Copy your dataset to the directory bucket.
3. Launch an EMR 7.13 cluster in the same Availability Zone.
4. Run your existing Spark workloads against the directory bucket and compare runtime.

Benchmark configuration

The following configuration was identical for both storage classes:

• Amazon EMR 7.13.0 with Apache Spark 3.5.6
• 8 core nodes (r8g.4xlarge, Graviton4) and 1 primary node
• 128 vCPUs, 1,024 GB memory across the cluster (dynamic allocation, Amazon EMR defaults)
• 924 GB of compressed Parquet data across 24 tables (3 TB scale factor)
• Same virtual private cloud (VPC), same Availability Zone (use1-az4), same Spark configuration
• AWS Glue Data Catalog as the metastore

Our results derived from the TPC-DS dataset are not directly comparable to official TPC-DS results due to differences in benchmark setup, configuration, and execution methodology.

Results

The total runtime across all 105 queries dropped by 31 percent, with S3 Express One Zone faster on 103 of the 105 individual queries.

For teams running hourly pipelines against multi-terabyte datasets, a 31 percent speedup can be the difference between finishing inside your processing window and missing it.

Speedup grows with data volume: 47% at 10 TB

We also ran the TPC-DS benchmark at 10 TB scale (4.1 TB of compressed Parquet) on the same 8-node cluster. For this larger dataset, we enabled maximizeResourceAllocation=true to fully utilize available memory. The S3 Express One Zone speedup grew from 31 percent at 3 TB to 47 percent at 10 TB.

On a fixed-size cluster, scaling the dataset increases the amount of data each node must read from Amazon S3. The workload becomes more I/O-bound, and S3 Express One Zone low-latency reads cut the wait time. The savings compound across thousands of tasks.

The following table summarizes both benchmark runs.

The pattern is clear: the more data your workload reads from Amazon S3, the more you benefit from S3 Express One Zone.

Biggest query speedups

The most I/O-heavy queries saw the largest gains, as detailed in the following table.

The two queries where S3 Standard was marginally faster (q8, q93) are both sub-5-second queries where the difference is within measurement noise (less than 1 second).

Cost analysis

Fast queries equate to less compute time and lower costs. We measured the cost of a single benchmark execution (all 105 queries), including Amazon Elastic Compute Cloud (Amazon EC2), Amazon EMR, and Amazon S3 request charges.

Same number of reads, lower latency

We enabled S3A IOStatistics logging (spark.hadoop.fs.iostatistics.logging.level=info) to capture the exact S3 API calls made during each run. Both storage classes issued nearly identical request counts, as detailed in the following table.

The request counts are virtually identical. The speedup comes entirely from S3 Express One Zone completing each read in 7 milliseconds instead of 121 milliseconds, a 17-fold reduction in per-request latency.

Compute cost

The cluster comprises nine instances (one primary and eight core, r8g.4xlarge). Amazon EMR bills per second.

At $0.00295 per second for the full cluster, the 31 percent runtime reduction with S3 Express One Zone saves $1.08 in compute charges per benchmark run.

At the time of writing, Amazon EC2 On-Demand pricing for r8g.4xlarge is $0.94256 per hour and the Amazon EMR rate is $0.23564 per hour (see Amazon EMR pricing for current rates).

Amazon S3 request cost

Following the April 2025 price reduction, S3 Express One Zone request pricing is up to 85 percent lower than S3 Standard for GET requests. With nearly identical request counts, the savings are significant, as shown in the following table.

At the time of writing, S3 Express One Zone request pricing is $0.00003 per 1,000 GET/HEAD requests and $0.00113 per 1,000 LIST requests. S3 Standard pricing is $0.0004 per 1,000 GET/HEAD requests and $0.005 per 1,000 LIST requests (see Amazon S3 pricing for current rates).

Total cost per run

The following table summarizes the total cost per run.

Each benchmark run costs 36 percent less on S3 Express One Zone, driven by both shorter compute time and significantly lower per-request pricing.

Storage cost in practice

S3 Express One Zone storage costs $0.11/GB/month compared to $0.023/GB/month for S3 Standard. However, most analytics pipelines don’t store data permanently in the hot tier. The recommended pattern is to write incoming data to both a directory bucket (S3 Express One Zone) and a general purpose bucket (S3 Standard) at ingestion time, then set an S3 Lifecycle expiration rule on the directory bucket to delete objects after a defined retention window.

For example, if the 924 GB dataset resides in S3 Express One Zone for 7 days before expiring, the incremental storage cost is approximately $23.72 for that window (924 GB × $0.11/30 × 7 days). During that week, every query against the dataset runs 31 percent faster and costs 36 percent less overall. Running this workload approximately two to three times a day recovers the storage premium within the retention window ($1.40 savings × 17 runs = $23.80). After expiration, the data remains available in S3 Standard for infrequent access.

This tiered approach works naturally for the following use cases:

• Ingestion pipelines that write to both tiers on arrival and query the S3 Express One Zone copy repeatedly before it expires
• Machine learning (ML) feature engineering that iterates on recent data in S3 Express One Zone during development, with the S3 Standard copy retained for long-term storage
• Time-series analytics where dashboards query the most recent 24–72 hours from S3 Express One Zone, with older data served from S3 Standard

Use S3 Lifecycle expiration rules on the directory bucket to automate cleanup. You pay the S3 Express One Zone premium only for the days your workloads are actively reading the data.

Clean up

To avoid ongoing charges after running this benchmark, complete the following steps:

1. Terminate your EMR cluster using the Amazon EMR console or AWS Command Line Interface (AWS CLI).
2. Delete objects in the directory bucket if you no longer need the benchmark data. S3 Express One Zone charges for storage by the hour.
3. Delete the directory bucket if it is no longer needed.
4. Review your AWS CloudTrail logs and Amazon CloudWatch metrics to confirm no resources remain active.

Conclusion

In this post, we set up an 8-node Graviton4 Amazon EMR cluster, ran the TPC-DS benchmark at 3 TB and 10 TB scale against both S3 Express One Zone and S3 Standard, and analyzed the performance and cost differences.

S3 Express One Zone delivers a 31 percent speedup for TPC-DS analytics at 3 TB scale on Amazon EMR, with most queries running faster. The benefit grows with data volume: at 10 TB, the speedup reaches 47 percent.

If your Spark workloads on Amazon EMR are I/O-bound, reading large Parquet datasets from Amazon S3, switching to a directory bucket can cut your runtime significantly. No application code changes are required. Co-locate your cluster with your directory bucket, disable fast partition discovery, and your workload benefits from single-digit millisecond read latency automatically.

For more information, see Getting started with S3 Express One Zone, the Amazon EMR Release Guide, and Upload data to Amazon S3 Express One Zone.