AWS Database Blog
PostgreSQL 18 on Amazon Aurora and Amazon RDS: Performance enhancements
Managing query performance across composite indexes, diagnosing memory spills in materialized CTEs, and upgrading major versions without plan regressions are everyday challenges for PostgreSQL users. PostgreSQL 18 addresses each of these directly with skip scan optimization that removes redundant indexes, EXPLAIN enhancements that expose storage behavior, and optimizer statistics that survive major version upgrades. You can try these features today on both Amazon Aurora PostgreSQL-Compatible Edition and Amazon Relational Database Service (Amazon RDS) for PostgreSQL.
This is Part 1 of a two-part series covering the key features in PostgreSQL 18. In this post, we focus on performance enhancements: skip scan optimization for multicolumn indexes, enhanced EXPLAIN output, automatic removal of unnecessary self-joins, and several vacuum and autovacuum improvements that help keep your database running efficiently. In Part 2, we cover security, monitoring, developer, and logical replication enhancements. For more information, see the PostgreSQL 18 release notes.
Prerequisites
You can read this post without running any commands, but if you want to try the examples, you must have these prerequisites:
- Create an Aurora PostgreSQL cluster or RDS for PostgreSQL instance if you don’t already have one. For instructions, see Create an Amazon Aurora PostgreSQL-Compatible DB cluster or Create a PostgreSQL DB instance, respectively.
- Create an Amazon Elastic Compute Cloud (Amazon EC2) instance to install the PostgreSQL client to access the Aurora PostgreSQL or RDS for PostgreSQL instance. For instructions, see Create your EC2 resources and launch your EC2 instance. Or you can set up connectivity between your RDS database and EC2 compute instance in 1-click.
- Install the PostgreSQL client. On Amazon Linux 2023, you can use the following commands to download the psql command line tool:
Install all required build dependencies:
Download PostgreSQL 18 source code. Replace
18.xin the commands below with the current minor version (for example, 18.4):Extract the source:
Navigate to the source directory:
Configure the build (set
bindirto/usr/binfor system-wide installation):Build and install only the client binaries, libraries, and headers:
Verify the installation:
Skip scan optimization for multicolumn indexes
PostgreSQL 18 introduces skip scan optimization, which allows the query planner to use multicolumn B-tree indexes more efficiently when the leading columns are not specified in the WHERE clause. This optimization can improve query performance in scenarios where you have composite indexes, but queries don’t always filter the leading columns.
Understanding the skip scan optimization
In previous PostgreSQL versions, a multicolumn index was used efficiently if the query included the leading column(s) in the WHERE clause. For example, with an index on (status, created_date), a query filtering only on created_date could not use the index.
Skip scan allows PostgreSQL to skip through the distinct values of the leading column(s) and use the index for subsequent columns, effectively treating the multicolumn index as multiple single-column indexes.
Example scenario
Let’s create a table with a multicolumn index and observe the skip scan optimization:
Now let’s run a query that filters only on created_date (not the leading column):
PostgreSQL 18 output (with skip scan):
In PostgreSQL 17 and earlier, this query would likely use a sequential scan or require a separate index on created_date:
PostgreSQL 17 output (without skip scan):
When skip scan is beneficial
Skip scan optimization is most effective when:
- The leading columns have low cardinality (few distinct values).
- The query filters on non-leading columns.
- The selectivity of the non-leading column filter is high.
Performance comparison
In our testing with Aurora PostgreSQL 17.10 and Aurora PostgreSQL 18.4, both on db.r6g.large with 1 million rows and 5 distinct values in the leading status column (warm buffer cache):
Without skip scan (Aurora PostgreSQL 17), the planner cannot use the composite index on (status, created_date) because the leading column is not in the WHERE clause. It falls back to a Parallel Seq Scan, reading all 7,353 blocks and finishing in 73.2 ms.
With skip scan (Aurora PostgreSQL 18), the planner uses the existing composite index with 11 index searches, reading only 2,266 blocks and finishing in 3.4 ms. That’s a 21x improvement with no additional index required.
The skip scan provides performance comparable to a dedicated single-column index without the storage overhead and maintenance cost of maintaining one.
Enhanced EXPLAIN
PostgreSQL 18 brings substantial improvements to the EXPLAIN command, making query performance analysis more intuitive and comprehensive. These enhancements provide deeper insights into query execution, resource usage, and optimization opportunities, helping developers and database administrators identify performance bottlenecks more effectively.
Automatic buffer statistics in EXPLAIN ANALYZE
OneOne of the significant usability improvements in PostgreSQL 18 is that EXPLAIN ANALYZE now includes buffer statistics by default, removing the need to explicitly specify the BUFFERS option. This change addresses a long-standing pain point where critical I/O information was hidden unless you remember to add the BUFFERS flag.
Prior to PostgreSQL 18, analyzing query performance required this syntax:
Without the BUFFERS option, you would see execution time and row counts but miss information about cache hits, disk reads, and I/O patterns. In PostgreSQL 18, buffer statistics are included automatically:
The output now shows:
The buffer statistics immediately reveal that most data was served from cache (18,234 shared buffer hits) with only 1,245 blocks read from disk. This helps you understand whether performance issues stem from I/O bottlenecks or other factors.
Memory and disk usage for materialized nodes
PostgreSQL 18 introduces storage visibility for materialized nodes in query plans. When the executor materializes a CTE for repeated access, the plan now reports whether the intermediate results were held in memory or spilled to disk, along with the peak storage consumed. This information was completely invisible in PostgreSQL 17 and earlier. You had to guess based on work_mem settings and row width estimates.
Example 1: Memory storage (default)
Output:
The line to focus on is Storage: Memory Maximum Storage: 296kB on the CTE Scan node. This tells you the entire materialized result set, all 4,948 rows, was held in RAM with a peak footprint of 296kB. No temporary files were created. No disk I/O was needed beyond reading the source table from shared buffers.
Example 2: Disk storage (low work_mem)
Now let’s see what happens when we artificially constrain work_mem so the materialized CTE can no longer fit in memory:
Output:
Notice the differences. Storage: Disk replaces Storage: Memory, and several new indicators appear. temp written=20 shows temporary file I/O occurred, and I/O Timings: temp write=0.127 quantifies the time spent writing to those temp files. Execution time increased from 3.057 ms to 3.268 ms, modest here because the dataset is small. On larger result sets, the penalty from disk spills compounds significantly.
Why this matters in practice
This storage reporting transforms how you diagnose and tune materialized CTEs. When a query that normally runs in milliseconds suddenly slows down, checking whether a previously in-memory CTE has flipped to Storage: Disk gives you an immediate explanation. The dataset grew past the memory threshold. The Maximum Storage value tells you exactly how much space the materialization needed, so you can make a targeted work_mem adjustment rather than guessing.
That said, the right response isn’t always to increase work_mem. Remember that work_mem is allocated per-operation and per-query within each session, so setting it too high on a system with many concurrent connections can lead to memory pressure. A better approach is to use the Maximum Storage value as a guide. If a CTE needs 296kB and your work_mem is 4 MB, you have plenty of room. If it needs 3.9 MB on a 4 MB setting, a modest bump to 8 MB keeps it in memory without being reckless. For queries where materialization overhead is the bottleneck, you might also consider whether the MATERIALIZED hint is truly necessary. Sometimes, letting the planner inline the CTE produces a better plan altogether.
The combination of Storage, Maximum Storage, temp written, and I/O Timings gives you a full picture of materialization cost that was unavailable before PostgreSQL 18. This turns what used to be an educated guess into a data-driven tuning decision.
Index searches metric in EXPLAIN ANALYZE
PostgreSQL 18 introduces a new metric in EXPLAIN ANALYZE output: Index Searches. This metric shows how many times the database traversed the index tree during query execution, providing valuable insights into index usage patterns.
What is the Index Searches counter?
The Index Searches counter appears in Index Scan, Bitmap Index Scan, and Index-Only Scan nodes. It reports the total number of index tree traversals across all node executions and loops.
Example: IN clause with multiple values
Consider a query searching for products with specific prices:
Output:
Notice Index Searches: 4 in the output. The query searches for 4 different price values, and PostgreSQL performed 4 separate index tree traversals, one for each value in the IN clause. Each search starts from the index root and navigates to the appropriate leaf page.
Why this matters
Before PostgreSQL 18, you saw that an index was used but not how many times the executor descended into the tree. Now you can. A high search count on an IN clause tells you each value triggered a separate root-to-leaf traversal. If those values happen to be contiguous, rewriting the predicate as a BETWEEN range might cut the number of searches down. In nested loop joins, the metric shows the total searches across all loop iterations. You can quickly tell whether the inner index is being hammered and whether a different join strategy would be cheaper.
The three enhancements covered here (automatic buffer statistics, materialized CTE storage and memory reporting, and index search counts) are only a subset of the EXPLAIN improvements shipped in PostgreSQL 18. For the full list of EXPLAIN-related enhancements, see the PostgreSQL 18 release notes.
Automatic removal of unnecessary self-joins
PostgreSQL 18 introduces Self-Join Elimination (SJE), a query planner optimization that automatically detects and removes inner joins of a table to itself when the join is probably redundant. This applies to SELECT queries, where the planner can safely determine that one side of the self-join contributes no additional rows or columns to the result and eliminate it entirely.
When a self-join is unnecessary
A self-join can be eliminated when two conditions are met:
- The join condition is on a unique or primary key column. This means at most one matching row per outer row, so the join can’t produce duplicates or filter rows.
- No columns from the inner table alias are referenced in the
SELECTlist orWHEREclause that couldn’t be satisfied by the outer alias alone.
When these conditions hold, the planner rewrites the query to scan the table once, removing the join entirely.
Example: ORM-generated self-join
Self-joins on primary keys are commonly generated by ORMs and query builders that compose queries from reusable fragments without awareness of redundancy. Consider an employees table with a primary key on id:
A query joining the table to itself on the primary key:
PostgreSQL 17 and earlier (executes a real join):
PostgreSQL 18 (join is eliminated, single scan):
The planner recognizes that joining employees to itself on the primary key id is redundant. Every row matches exactly one row in the other alias, and that row is the same physical row. The join is rewritten to a simple scan.
With a WHERE clause
SJE also works when additional filters are present:
Again, a single scan. The self-join is eliminated and the filter is applied directly.
When self-join elimination does not apply
SJE requires a unique or primary key join condition. A self-join on a non-unique column, such as finding colleagues in the same department, can’t be eliminated because multiple rows can match:
Here the join is preserved. dept is not unique, so one employee can match many colleagues, and the join genuinely changes the result set.
Scope: SELECT only
The PostgreSQL 18 implementation applies SJE only to SELECT queries. Self-joins in UPDATE, DELETE, and MERGE statements aren’t eliminated. This is intentional. DML queries involve EvalPlanQual (EPQ) re-evaluation for concurrent row updates, where the two table aliases might need to reference different row versions (snapshot compared to latest). Eliminating the join in that context would produce incorrect results. The PostgreSQL community has scoped the initial implementation conservatively, with DML support planned for a future release.
Fine-tuned autovacuum control with autovacuum_vacuum_max_threshold
PostgreSQL 18 introduces autovacuum_vacuum_max_threshold, a new parameter that caps the number of dead tuples that can accumulate before autovacuum triggers a VACUUM, regardless of table size. This addresses a long-standing problem with very large tables where the scale-factor-based formula can delay autovacuum for far too long.
The problem with large tables
Autovacuum decides when to vacuum a table by comparing the number of updated or deleted tuples against a threshold calculated as:
With the defaults (threshold = 50, scale_factor = 0.2), this works well for small and medium tables. But for very large tables, the formula produces thresholds that allow enormous amounts of bloat to accumulate before autovacuum fires:
| Table size | Dead tuples needed to trigger autovacuum (before PG18) |
| 1 million rows | 200,050 |
| 100 million rows | 20,000,050 |
| 500 million rows | 100,000,050 |
| 1 billion rows | 200,000,050 |
On a 1 billion row table, autovacuum waits for 200 million dead tuples before running. That’s significant table bloat, degraded query performance, and increased risk of transaction ID wraparound pressure.
The workaround before PG18 was to manually set autovacuum_vacuum_scale_factor = 0 and autovacuum_vacuum_threshold = N per table, but this required identifying every large table and maintaining those settings as tables grew.
How autovacuum_vacuum_max_threshold works
PostgreSQL 18 changes the formula to:
The new parameter acts as a ceiling. When the scale-factor calculation exceeds the max threshold, autovacuum fires at the cap instead. The default is 100,000,000 tuples, meaning no table will accumulate more than 100 million dead tuples before autovacuum runs, regardless of how large the table is.
For the 1 billion row table example, the effective threshold is now capped at 100 million instead of 200 million.
Configuring in Amazon RDS for PostgreSQL and Aurora PostgreSQL
autovacuum_vacuum_max_threshold is a dynamic parameter (sighup-level). It can be changed in your parameter group and applied without a reboot.
Amazon RDS for PostgreSQL: Modify the DB instance parameter group.
Aurora PostgreSQL: Modify the DB cluster parameter group (applies to all instances in the cluster).
In both cases, the parameter change is applied without a reboot. You can verify using SHOW autovacuum_vacuum_max_threshold; from any connected session.
Lower the global cap for all tables:
Override per table:
Disable the global cap entirely:
Choosing the right value
The default of 100 million is a reasonable safety net for most workloads. For databases with large tables with high churn across both old and new rows, lowering it to 10 million can help autovacuum stay ahead of dead tuple accumulation while balancing for freeze activity. For append-only or read-heavy tables where dead tuples accumulate slowly, the default or even -1 (disabled) may be appropriate.
You can check the current effective threshold for any table by querying pg_stat_user_tables alongside pg_class:
vacuum_truncate server variable
PostgreSQL 18 introduces vacuum_truncate as a server-wide GUC (Grand Unified Configuration) parameter. This gives database administrators a single control point to disable VACUUM’s file truncation behavior across all tables, without modifying individual table storage options.
Understanding VACUUM truncation
When VACUUM runs, it goes through multiple phases, including scanning the heap, vacuuming indexes and the heap, and eventually truncating the heap. The truncation phase removes empty pages from the end of a table data file and returns that disk space to the operating system. While this sounds beneficial, it comes with an important cost: truncation requires an AccessExclusiveLock on the table, the same lock level used by ALTER TABLE, DROP TABLE, and similar DDL operations.
During the first three phases, VACUUM holds only a ShareUpdateExclusiveLock, which allows concurrent reads and writes. But when it enters the truncation phase, it escalates to AccessExclusiveLock, briefly blocking all other access to the table. On busy OLTP systems with large tables, this lock escalation can cause noticeable query latency spikes.
Impact on read replicas
The truncation lock has a particularly significant effect on Aurora PostgreSQL and RDS for PostgreSQL read replicas. When the primary instance truncates a table, it writes an XLOG_STANDBY_LOCK record to the WAL stream. The replica’s WAL replay process must acquire the same AccessExclusiveLock on the relation before it can apply this record. If a long-running query on the replica is holding a conflicting lock, the replica will cancel that query after a 30-second wait in order to proceed with WAL replay. Users on the read replica see:
This is a common source of unexpected query cancellations on read replicas, particularly when autovacuum is actively truncating large tables on the primary.
Before PostgreSQL 18: per-table control only
Prior to PostgreSQL 18, vacuum_truncate existed only as a storage-level parameter set per table:
This approach required identifying every table where truncation was causing issues and modifying each one individually, a maintenance burden in databases with hundreds of tables.
PostgreSQL 18: server-wide GUC
PostgreSQL 18 promotes vacuum_truncate to a server-wide GUC, so you can control truncation behavior globally through your parameter group. The default value remains on (truncation enabled), preserving backward compatibility.
Check the current setting:
Configuring vacuum_truncate in Amazon RDS for PostgreSQL
To disable truncation globally, create a custom parameter group with vacuum_truncate = off and apply it to your instance. This is a dynamic parameter and no reboot is required. You can verify the setting after the reboot:
Demonstrating the behavior difference
You can observe the effect of vacuum_truncate directly in the VACUUM verbose output, specifically how many pages were removed, how many remain, and whether a truncation line appears.
With vacuum_truncate = on (default):
The line truncated 4425 to 45 pages confirms that VACUUM removed 4,380 empty pages and returned the space to the OS. This truncation requires an AccessExclusiveLock.
With vacuum_truncate = off:
The truncation line is completely absent. Dead tuples are still removed (990,000 rows cleaned up), but the empty pages remain in the file. No AccessExclusiveLock is acquired, and no WAL truncation record is sent to replicas.
When to use vacuum_truncate = off
Disabling truncation is appropriate when:
- Read replicas experience frequent query cancellations because of
canceling statement due to conflict with recoveryerrors caused byVACUUMtruncation on the primary. - Tables have high write churn where deleted space will quickly be reused by new inserts. Truncating and then re-extending the file wastes I/O.
- Large tables on busy OLTP systems where even a brief
AccessExclusiveLockcauses visible latency spikes.
Keep truncation enabled (the default) when:
- Tables are periodically bulk-deleted and the space genuinely won’t be reused, truncation returns disk space to the OS.
- Storage costs matter and you want
VACUUMto actively reclaim space. - No read replicas are present, removing the WAL conflict concern.
Per-table override
The server-wide GUC can still be overridden at the table level. If you set vacuum_truncate = off globally but want truncation enabled for a specific table:
Conversely, if the global setting is on but a specific high-traffic table needs truncation disabled:
This layered control (server default using parameter group, per-table override using storage options) gives you precise control over truncation behavior across your database.
autovacuum_worker_slots
PostgreSQL 18 introduces autovacuum_worker_slots, a new parameter that separates the concept of allocating autovacuum worker slots from using them. This separation allows autovacuum_max_workers to be adjusted at runtime without a server restart, a significant operational improvement for tuning autovacuum under changing workloads.
The problem before PostgreSQL 18
Before PostgreSQL 18, autovacuum_max_workers was the only parameter controlling autovacuum concurrency. Because it directly controlled how many backend slots were reserved at startup, it had a postmaster context, meaning any change required a full server restart. If you needed to temporarily increase autovacuum workers to catch up on table bloat during a high-write period, you had no choice but to restart the instance.
How it works in PostgreSQL 18
PostgreSQL 18 introduces a two-level model:
autovacuum_worker_slotssets the maximum number of backend slots reserved for autovacuum workers at startup. This is apostmaster-level parameter and still requires a restart to change. It acts as a hard ceiling.autovacuum_max_workerscontrols how many of those slots are actually used at any given time. It’s now a dynamic parameter (sighupcontext), meaning it can be changed at runtime by reloading the configuration, and no restart required.
By default, autovacuum_worker_slots is 16 and autovacuum_max_workers is 3. You can increase autovacuum_max_workers up to 16 at any time without restarting the instance.
Configuring in Amazon RDS for PostgreSQL
In RDS and Aurora PostgreSQL, you can update autovacuum_max_workers in your parameter group with Apply immediately, no reboot needed. To increase the ceiling (autovacuum_worker_slots), update the parameter group and reboot the instance.
Example: temporarily increase autovacuum workers during a high-write period
Update autovacuum_max_workers in your parameter group from 3 to 6, apply immediately, and verify:
Revert it back when the workload normalizes.
Important: autovacuum_max_workers can’t exceed autovacuum_worker_slots. If you attempt to set it higher, PostgreSQL caps it at the slots value. To allow higher autovacuum_max_workers, first increase autovacuum_worker_slots (which requires a restart), then adjust autovacuum_max_workers freely within the new ceiling.
Per-table vacuum and analyze timing
PostgreSQL 18 adds four new columns to pg_stat_all_tables (and its variants pg_stat_user_tables, pg_stat_sys_tables) that track the cumulative time spent on maintenance operations per table:
| Column | Description |
total_vacuum_time |
Cumulative time spent in manual VACUUM operations (milliseconds) |
total_autovacuum_time |
Cumulative time spent in autovacuum operations (milliseconds) |
total_analyze_time |
Cumulative time spent in manual ANALYZE operations (milliseconds) |
total_autoanalyze_time |
Cumulative time spent in autoanalyze operations (milliseconds) |
Before PostgreSQL 18, pg_stat_user_tables showed when the last vacuum or analyze ran (last_vacuum, last_autovacuum) but gave no indication of how long it took. Identifying which tables consumed the most maintenance time required either parsing log files or using pg_stat_progress_vacuum during an active operation. These new columns make that information permanently available and cumulative.
Example
Create a table with 5 million rows, delete half of them, then run VACUUM VERBOSE and ANALYZE VERBOSE:
Now query the new timing columns:
The elapsed: 0.18 s from VACUUM VERBOSE maps directly to 189ms in total_vacuum_time, and elapsed: 0.10 s from ANALYZE VERBOSE maps to 107ms in total_analyze_time. The view accumulates these values across every manual and automatic operation, giving you a running total of maintenance time per table.
Because these columns are cumulative, running VACUUM and ANALYZE again adds to the totals, giving you a running picture of maintenance overhead over the lifetime of the table.
Practical use cases
Identify tables with high maintenance overhead
Compare manual and automatic maintenance time
Tables where total_autovacuum_time is very high relative to total_vacuum_time might benefit from tuning autovacuum thresholds. Tables where both are high are candidates for investigating bloat, index health, or write patterns.
Clean up resources
If you created database objects while following along with this post, run the following commands to remove them:
If you created an Aurora PostgreSQL cluster or RDS for PostgreSQL instance specifically for this walkthrough and no longer need it, delete the instance to avoid ongoing charges. For instructions, refer to Deleting a DB instance or Deleting an Aurora DB cluster. Similarly, terminate any Amazon EC2 instance that you created solely for the PostgreSQL client connection.
Conclusion
With skip scan, your existing composite indexes now handle queries they previously could not without maintaining additional indexes. Enhanced EXPLAIN turns CTE tuning from guesswork into data-driven decisions. Self-join elimination automatically fixes a class of ORM-generated inefficiency. And the new autovacuum controls help keep your largest tables healthy without manual per-table configuration. These improvements are available today on both Aurora PostgreSQL-Compatible Edition and Amazon RDS for PostgreSQL.
In Part 2, we cover security enhancements such as MD5 password support, monitoring improvements for parallel workers and replication, the new uuidv7() function for developers, optimizer statistics preservation during upgrades, and logical replication enhancements. To get started, create an Aurora PostgreSQL DB cluster or an RDS for PostgreSQL DB instance on PostgreSQL 18 and try these features today.