Accelerate Spark on EMR Serverless with larger workers and shuffle-optimized disks

With Amazon EMR Serverless, you can run open source big data frameworks such as Apache Spark and Apache Hive without managing clusters or infrastructure. Customers are increasingly choosing EMR Serverless for their analytics workloads because of the simplicity of a fully managed, serverless experience. As adoption grows, teams want to bring their most demanding jobs to Serverless too. These jobs include large-scale joins, shuffle-heavy ETL, and memory-intensive analytics that previously required carefully sized clusters. Customers migrating these heavyweight workloads from their Spark clusters often need the same compute shapes on EMR Serverless to achieve the same price-performance and make migration easier.

Today, we’re excited to announce a new 32 vCPU / 244 GB worker configuration on Amazon EMR Serverless, giving you the headroom to run your most intensive workloads without leaving the serverless experience.

Overview of larger workers

An EMR Serverless application uses workers to run your Spark tasks, and you can choose a worker size that matches your workload. The new 32 vCPU worker offers a large compute and memory footprint (32 vCPUs and 244 GB of memory) that supports attaching up to 2,000 GB of shuffle-optimized disk. This combination benefits three common workload patterns:

• Shuffle-intensive workloads – Wide transformations such as join, groupBy, sortBy, and repartition redistribute large amounts of data across the cluster. Larger workers keep more shuffle data local to each executor and read and write shuffle blocks on higher-throughput disk, reducing remote fetches and shuffle wait time.
• I/O-heavy workloads – Queries that scan large datasets or spill intermediate data to disk are limited by disk throughput and IOPS. The shuffle-optimized disk raises the ceiling on both sides. Large workers also improve network bandwidth.
• Memory-intensive workloads – Higher per-executor memory (244 GB versus 30 GB) lets more data be cached and processed in memory without spilling, which helps with data skew and caching.

Benchmark setup

We compared the recommended large-worker configuration against common small-worker defaults at identical total compute (192 vCPUs).

• Large workers – 6 executors × (32 vCPU / 244 GB / 2,000 GB shuffle-optimized disk).
• Small workers – 48 executors × (4 vCPU / 30 GB / 200 GB standard disk).

The two configurations differ in both worker shape and disk class. We compare them as paired configurations, since shuffle-optimized disk is the intended disk type for the 32 vCPU worker. The reported gains reflect this combined effect.

Both configurations set spark.dynamicAllocation.enabled=false and spark.scheduler.minRegisteredResourcesRatio=1 to reduce variance from worker launch times. Requiring full registration ensures each query starts only after the cluster is ready. We ran every query with 3 iterations and reported the median. Both benchmarks ran on EMR release emr-7.13.0 in a virtual private cloud (VPC) with an Amazon Simple Storage Service (Amazon S3) gateway endpoint attached to the private subnets.

Benchmark results

This post presents benchmark results comparing the new 32 vCPU workers against the existing 4 vCPU standard workers using the industry-standard TPC-DS and TPC-H benchmarks. Across 126 queries (104 TPC-DS and 22 TPC-H), large workers delivered an average of 29% faster query execution and 29% lower query-attributed cost, with zero regressions and improvements peaking at 45–55% on shuffle-heavy, multi-table join queries. Both the larger executor shape and the shuffle-optimized disk contribute to these gains. The disk advantage is most pronounced on the shuffle and I/O-heavy queries where the largest improvements appear.

The following table summarizes the results across both benchmarks. Large workers won every query on both performance and cost.

Note: The benchmark results in this post are derived from the TPC-DS and TPC-H benchmark specifications. TPC-DS and TPC-H are trademarks of the Transaction Processing Performance Council.

TPC-DS 3 TB benchmark

TPC-DS is an industry-standard decision support benchmark that models complex analytical workloads with multi-table joins, subqueries, and aggregations. We ran 104 queries from the TPC-DS v2.4 suite against a 3 TB partitioned Parquet dataset, with 3 iterations per query for statistical confidence.

The test environment was as follows:

• Dataset: 3 TB partitioned Parquet (24 TPC-DS tables).
• Queries: 104 (full suite minus 4 incompatible with the dataset schema).
• Iterations: 3 per query (one query per start-job-run, a fresh Spark application each time).
• Networking: VPC with an Amazon S3 gateway endpoint.
• EMR release: emr-7.13.0.

The following chart shows the top 10 and bottom 5 queries by performance improvement. All 104 queries show a positive improvement, with the largest gains on shuffle-heavy queries such as q58 (45%), q21 (43%), and q12 (42%).

Performance improvement for the top 10 and bottom 5 TPC-DS queries, 32 vCPU compared to 4 vCPU standard workers

Why large workers are faster for TPC-DS

TPC-DS queries are characterized by complex multi-table joins that generate large shuffle operations. With 6 large executors instead of 48 small ones, the shuffle-optimized disk provides significantly higher random I/O throughput for reading and writing shuffle blocks. In addition, fewer executors mean less network coordination during shuffle. Each executor fetches shuffle data from only 5 remote sources instead of 47. This increases the share of shuffle data read locally, which improves performance.

TPC-H 1 TB benchmark

TPC-H is a decision support benchmark that focuses on ad hoc analytical queries. We ran all 22 TPC-H queries against a 1 TB dataset, with 3 iterations per query. Each query was submitted as a separate start-job-run (a fresh Spark application) to simulate the realistic pattern of independent ad hoc queries arriving without session warmup.

The test environment was as follows:

• Dataset: 1 TB partitioned Parquet.
• Queries: 22.
• Iterations: 3 per query (one query per start-job-run, a fresh Spark application each time).
• Networking: VPC with an Amazon S3 gateway endpoint.
• EMR release: emr-7.13.0.

Performance improvement for TPC-H queries, 32 vCPU compared to 4 vCPU standard workers

Why large workers are faster for TPC-H

The shuffle-optimized disk accelerates this initial table-scan phase. The subsequent query execution benefits from higher per-executor memory (244 GB versus 30 GB), which lets more data be processed in memory without spilling to disk.

Cost calculation and improvement results

The following table compares the cost of running TPC-DS and TPC-H benchmarks on larger workers (32 vCPU / 244 GB / 2,000 GB shuffle-optimized disk) versus smaller workers (4 vCPU / 30 GB / 200 GB standard disk) on EMR Serverless. Both configurations use identical total compute (192 vCPUs).

TPC-DS 3 TB

TPC-H 1 TB

Notes:

• Runtime represents the sum of median query execution times across all queries in the benchmark (3 iterations per query, median reported).
• Calculated cost is computed using EMR Serverless on-demand pricing: vCPU-hr ($0.052624), Memory GB-hr ($0.0057785), Storage GB-hr ($0.000111).
• Illustrative TPC-DS run cost calculation:
  • Large worker: (148.31 * $0.052624) + (1,130.87 * $0.0057785) + (9,269.47 * $0.000111) = $15.37.
  • Smaller worker: (195.74 * $0.052624) + (1,468.07 * $0.0057785) + (9787.15 * $0.000111) = $19.87.
• Cost is proportionally attributed to query execution time, excluding Spark initialization and shutdown overhead.
• Both configurations use identical total vCPU (192) with Dynamic Resource Allocation disabled.
• The improvement percentage represents the cost reduction achieved by larger workers relative to smaller workers.
• Disk Sizing: The larger workers provisioned 25% more disk (12 TB versus 9.6 TB), yet the total cost came out 22.6–37.2% lower. Disk is by far the lowest-priced billing dimension on EMR Serverless ($0.000111 per GB-hour, versus 52x that for memory and 474x for vCPU) and made up only $1.03 of the $15.37 TPC-DS total. Many customers under-provision disk to trim this smallest line item, and it backfires. Shuffles slow down, jobs run longer, and every extra second is billed on the costlier vCPU and memory dimensions. On large workers (8+ vCPUs), shuffle-optimized disks scale IOPS and throughput with capacity, which helps move shuffle data faster. Size disk as a performance lever, not a cost lever.

When to use large workers

To determine whether the 32 vCPU worker with shuffle-optimized disk will benefit your Spark applications, consider the following:

• Check the Stages tab of the Spark History Server for your EMR Serverless application and review the Shuffle Read and Shuffle Write columns. The larger the shuffle volume relative to the number of executors, the more a job benefits from keeping shuffle data local on high-throughput disk. Jobs that shuffle tens of gigabytes or more per executor are strong candidates.
• Check the Stages tab for the Spill (memory) and Spill (disk) columns and the Executors tab for peak JVM memory. If data is spilling to disk or peak memory is close to the configured executor memory, the higher memory of the large worker can remove the spill and improve performance.

When not to use large workers

Large workers are not the best fit for every workload:

• I/O-bound jobs – For workloads whose runtime is dominated by reading and writing data (rather than shuffle or memory pressure), a larger number of smaller workers (for example, 8 or 16 vCPU) with the same disk sizes can deliver better aggregate throughput. Spreading the work across more executors increases read/write parallelism, although a few large workers can leave disk and network bandwidth underused.

Conclusion

In this post, we demonstrated that EMR Serverless 32 vCPU workers deliver performance and cost benefits for shuffle-intensive Spark workloads. Across 126 TPC-DS and TPC-H queries, larger workers achieved an average 29% faster execution and 29% lower cost.

We recommend evaluating the 32 vCPU worker with shuffle-optimized disk for your shuffle-intensive and I/O-heavy EMR Serverless Spark workloads. To get started, use the following configuration:

--conf spark.executor.cores=32
--conf spark.executor.instances=6
--conf spark.executor.memory=220g
--conf spark.emr-serverless.executor.disk=2000G
--conf spark.emr-serverless.executor.disk.type=SHUFFLE_OPTIMIZED

For more information about worker configurations, refer to Worker configurations in the Amazon EMR Serverless User Guide. We also recommend staying up to date with the latest EMR releases to take advantage of ongoing performance improvements.

About the authors