Migrate Amazon Aurora PostgreSQL across major versions with active Debezium CDC connectors using native logical replication

Organizations running Debezium CDC connectors on Amazon Aurora PostgreSQL-Compatible Edition face a specific challenge when upgrading database major versions. For most upgrades, an in-place major version upgrade or an Amazon Aurora blue/green deployment is the recommended approach, and we suggest you start there. But both can disrupt the change data capture (CDC) event pipelines that downstream systems depend on. An in-place major version upgrade requires downtime that interrupts active CDC consumers, and a blue/green deployment drops logical replication slots at switchover. That forces Debezium connectors into a full re-snapshot, which can take hours per database. For teams operating dozens of clusters with active CDC connectors, that re-snapshot can create significant risk and operational burden.

This post is for customers who can’t tolerate a full connector re-snapshot during an upgrade. We show how to perform a migration with upgrade using PostgreSQL native logical replication to bridge a source cluster (your current version) and a separate target cluster (the new major version), rather than upgrading a single cluster in place. When you follow the documented procedure, you can cut your Debezium connectors over to the new version while helping to minimize the risk of data loss. In our testing, the database-side cutover completed in seconds rather than minutes, and Kafka consumers saw a brief pause followed by resumed event delivery. We also cover the failure scenarios you should plan for and the post-cutover monitoring you need to detect them.

Solution overview

This pattern uses Aurora PostgreSQL native logical replication, which you configure to synchronize data from a source cluster (PostgreSQL 14 in this walkthrough) to a target cluster on a newer major version (PostgreSQL 17 in our example). You can apply the same pattern to any supported major-version pair. Your existing Debezium connector continues operating on the source throughout this phase. Once the target is fully synchronized, you execute a brief cutover sequence: you disable the replication subscription, synchronize sequences, create a new Debezium replication slot on the target, and redirect the connector.

Figure 1: Native logical replication bridges the source PostgreSQL 14 and target PostgreSQL 17 clusters, and the Debezium connector swaps from the source slot to a target slot at cutover while keeping the same topic prefix

You create the target cluster’s Debezium slot at the current log sequence number (LSN), so you can start the connector with no backlog to process. When you set snapshot.mode=never, Debezium skips the initial snapshot and reads only new changes from the slot. Kafka consumers continue reading from the same topics with the same topic.prefix. They observe a brief pause, then events resume.

This post builds on the AWS guidance for using logical replication to perform a major version upgrade for Aurora PostgreSQL. That documentation walks through the database to upgrade itself. This post focuses on the operational tasks the documentation doesn’t cover: keeping active Debezium CDC connectors streaming across the upgrade, including how you swap the connector at cutover without a re-snapshot, synchronize sequences, and handle failure scenarios such as connector stalls and in-flight schema changes.

Network considerations

This pattern requires low-latency connectivity between source and target clusters. In our testing, we placed the clusters in the same virtual private cloud (VPC) and Availability Zone. Cross-AZ replication typically adds only single-digit milliseconds of round-trip latency and remains viable. Measure the actual latency in your environment before relying on it. Cross-Region replication isn’t recommended, because the added latency raises the risk of lag accumulation during the synchronization phase.

Prerequisites

Before you begin, verify the following:

• Source cluster: Aurora PostgreSQL with rds.logical_replication (static parameter, needs rebooting to take effect) set to 1 in the cluster parameter group (custom parameter group is required) . This walkthrough uses Aurora PostgreSQL 14 as the source. The pattern works on any major version that supports logical replication.
• Target cluster: A newer Aurora PostgreSQL major version, provisioned in the same VPC as the source with cross-cluster security group access. This walkthrough upgrades to Aurora PostgreSQL 17 because that was the target in our scenario, but any supported newer major version works (for example, 14 to 15 or 15 to 16).
• Primary keys: Make sure your tables participating in CDC have a primary key (or an explicitly set REPLICA IDENTITY). Logical replication uses this replica identity to uniquely match rows when applying UPDATE and DELETE events on the target. To find tables in your CDC schema that lack a primary key:

SELECT n.nspname AS schema, c.relname AS table_name
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind = 'r' AND n.nspname = 'test_cdc'
  AND NOT EXISTS (SELECT 1 FROM pg_index i WHERE i.indrelid = c.oid AND i.indisprimary);

For any table returned, add a primary key or set an explicit REPLICA IDENTITY before you proceed.

• Debezium: Version 2.x or later with the PostgreSQL connector, running on Amazon Managed Streaming for Apache Kafka (Amazon MSK) Connect, Amazon Elastic Container Service (Amazon ECS), or self-managed Kafka Connect.
• Permissions: A database user holding the rds_replication role on both clusters (the Amazon Relational Database Service (Amazon RDS) and Aurora role that grants the privileges needed to manage and stream from logical replication slots), plus CREATE privilege on the target database.
• Tools: psql or equivalent SQL client, pg_dump for schema export, and access to the Kafka Connect REST API or the AWS Management Console.

Parameter group settings

Configure the following parameters on both source and target cluster parameter groups:

Treat the Recommended starting point column as the guidance to follow. The defaults are shown for reference and (for max_logical_replication_workers and max_worker_processes) are computed by the engine rather than fixed numbers. These are starting points, not fixed values. Size them to your workload. The bridge subscription needs one replication slot and one write-ahead log (WAL) sender beyond what your existing Debezium connectors already consume, so set max_replication_slots and max_wal_senders with headroom above your current usage. max_logical_replication_workers and max_worker_processes govern how many tables sync in parallel during the initial copy. Raise them if you have many tables to synchronize.

The parameter max_slot_wal_keep_size is a safety guard against unbounded WAL growth. If your replication slot goes inactive during the transition (for example, subscriber downtime or a paused bridge), the system accumulates WAL files indefinitely by default. Setting this cap reclaims storage even if a slot stalls. You can monitor the replication slot lag through the Amazon CloudWatch metric OldestReplicationSlotLag. This is a dynamic parameter and can be applied without a reboot.

If the source database handles large transactions or sustained high write volume, consider increasing the value of rds.logical_wal_cache. This write-through cache minimizes reliance on the Aurora storage layer. Instead of consistently writing to and reading from this layer, Aurora PostgreSQL uses a buffer to cache the logical WAL stream during the replication process, which reduces the need to access disk. You can monitor the metric with the following SQL query:

SELECT * FROM aurora_stat_logical_wal_cache();

Note: A few of the preceding parameters are static and require a cluster reboot to take effect. Plan the change during a maintenance window before beginning the activity.

To configure the source cluster for logical replication

1. Verify that logical replication is enabled on the source:

SHOW rds.logical_replication;
SHOW wal_level;
-- wal_level returns "logical"

2. Create a publication that includes tables participating in CDC. On PostgreSQL 14, use an explicit table list (the FOR ALL TABLES IN SCHEMA syntax requires PostgreSQL 15 or later):

CREATE PUBLICATION debezium_pub FOR TABLE
    test_cdc.orders,
    test_cdc.customers,
    test_cdc.inventory;

3. Verify the publication includes the expected tables:

SELECT schemaname, tablename
FROM pg_publication_tables
WHERE pubname = 'debezium_pub';

4. Verify your existing Debezium replication slot is active and consuming:

SELECT slot_name, plugin, active, restart_lsn, confirmed_flush_lsn
FROM pg_replication_slots
WHERE slot_name = 'debezium_slot';

To create and configure the target cluster

1. Create the target Aurora PostgreSQL cluster on the major version you want:

aws rds create-db-cluster \
  --db-cluster-identifier pg17-target \
  --engine aurora-postgresql \
  --engine-version 17.4 \
  --master-username admin \
  --master-user-password <PASSWORD> \
  --vpc-security-group-ids <SG_ID> \
  --db-subnet-group-name <SUBNET_GROUP> \
  --db-cluster-parameter-group-name <PG17_PARAM_GROUP>

Here, --master-username and --master-user-password are the literal Amazon RDS API parameter names that set the cluster’s administrative user and password.

2. Create a writer instance in the cluster:

aws rds create-db-instance \
  --db-instance-identifier pg17-target-instance \
  --db-cluster-identifier pg17-target \
  --db-instance-class db.r6g.large \
  --engine aurora-postgresql

Choose an instance class that matches or exceeds your source cluster’s capacity, because the target will eventually serve production traffic. This example uses db.r6g.large for demonstration. Size based on your workload. Wait for the cluster and instance to reach available status before proceeding:

aws rds wait db-instance-available --db-instance-identifier pg17-target-instance

3. Export the schema from the source and apply it to the target. Before you apply the schema, install any PostgreSQL extensions your schema depends on (for example, pgcrypto) on the target, because pg_dump --schema-only emits CREATE EXTENSION statements that fail if the extension is not already available on the target cluster:

# On the target, pre-install any required extensions, for example:
# psql -h <TARGET_ENDPOINT> -U admin -d testdb -c "CREATE EXTENSION IF NOT EXISTS pgcrypto;"

pg_dump -h <SOURCE_ENDPOINT> -U admin -d testdb \
  --schema-only --schema=test_cdc > schema.sql

psql -h <TARGET_ENDPOINT> -U admin -d testdb -f schema.sql

4. Verify that all tables exist on the target with matching structures:

SELECT table_name, column_name, data_type
FROM information_schema.columns
WHERE table_schema = 'test_cdc'
ORDER BY table_name, ordinal_position;

5. Create the replication user (used by the logical replication subscription) and the Debezium user (used by the connector) on the target, if you aren’t using the admin user:

-- Subscription owner for logical replication:
CREATE USER replication_user WITH PASSWORD '<PASSWORD>';
GRANT rds_replication TO replication_user;
GRANT ALL ON SCHEMA test_cdc TO replication_user;
GRANT ALL ON ALL TABLES IN SCHEMA test_cdc TO replication_user;

-- Debezium connector user (referenced by database.user in the connector config):
CREATE USER debezium_user WITH PASSWORD '<PASSWORD>';
GRANT rds_replication TO debezium_user;
GRANT USAGE ON SCHEMA test_cdc TO debezium_user;
GRANT SELECT ON ALL TABLES IN SCHEMA test_cdc TO debezium_user;

To establish logical replication

1. On the target cluster, create a subscription that connects to the source:

CREATE SUBSCRIPTION pg14_to_pg17_sub
    CONNECTION 'host=<SOURCE_ENDPOINT> port=5432 dbname=testdb user=replication_user password=<PASSWORD>'
    PUBLICATION debezium_pub
    WITH (
        copy_data = true,
        create_slot = true,
        enabled = true,
        synchronous_commit = 'off'
    );

Setting synchronous_commit = 'off' on the subscription speeds up the initial copy and steady-state apply by not waiting for the target’s local flush on every transaction. This is appropriate here because the source remains the system of record until cutover. It is a subscription-level setting for this migration, not a change to the source database’s durability.

2. On the target, verify that all tables have reached the ready state. PostgreSQL logical replication first copies a snapshot of existing rows (the initial data copy), then switches each table to streaming ongoing changes; srsubstate = 'r' (“ready”) means a table has finished the copy and is now streaming:

SELECT srsubstate, srrelid::regclass AS table_name
FROM pg_subscription_rel
WHERE srsubid = (SELECT oid FROM pg_subscription WHERE subname = 'pg14_to_pg17_sub');
-- Each row returns srsubstate = 'r'

Each replicated table reports r (ready) once its initial copy is complete:

 srsubstate | table_name
------------+-------------------
 r          | test_cdc.orders
 r          | test_cdc.customers
 r          | test_cdc.inventory

3. On the source, monitor the subscription’s replication lag until it reaches zero:

SELECT slot_name,
       pg_current_wal_lsn() AS current_lsn,
       confirmed_flush_lsn,
       pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn) AS lag_bytes
FROM pg_replication_slots
WHERE slot_name LIKE 'pg14_to_pg17%';

As the target catches up, lag_bytes trends toward zero:

     slot_name     | current_lsn  | confirmed_flush_lsn | lag_bytes
-------------------+--------------+---------------------+-----------
 pg14_to_pg17_sub  | 0/4A12F8C0   | 0/4A12F8C0          |         0

4. Validate data consistency between source and target by comparing row counts:

-- Run on both source and target:
SELECT 'orders' AS tbl, COUNT(*) FROM test_cdc.orders
UNION ALL SELECT 'customers', COUNT(*) FROM test_cdc.customers
UNION ALL SELECT 'inventory', COUNT(*) FROM test_cdc.inventory;

5. Verify that ongoing data manipulation language (DML) operations replicate correctly by inserting a test row on the source and checking it appears on the target:

-- On source:
INSERT INTO test_cdc.orders (customer_id, product_name, amount, status)
VALUES (99999, 'REPLICATION_TEST', 1.00, 'test');

-- On target (after a few seconds):
SELECT * FROM test_cdc.orders WHERE product_name = 'REPLICATION_TEST';

To prepare for cutover

Perform these steps during a scheduled maintenance window. The subscription should show zero lag before you proceed.

1. Verify that replication lag is consistently at or near zero:

-- On source:
SELECT pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn) AS lag_bytes
FROM pg_replication_slots
WHERE slot_name LIKE 'pg14_to_pg17%';
-- Returns 0 or near-0 consistently over several checks

Proceed only when lag_bytes holds at or near zero across several consecutive checks. If it will not drain (for example, it plateaus at a non-trivial value or keeps growing), do not begin the cutover: a non-draining subscription means the target is not fully caught up, so cutting over would risk losing the un-replicated changes. Set an abort threshold that fits your workload. For example, if lag hasn’t reached near-zero within a few minutes of write traffic quiescing, stop and investigate before retrying (a common cause is a long-running or idle-in-transaction session on the source holding back the stream, covered in Scenario 5).

2. Prepare the Debezium connector configuration for the target (copy from your existing source connector, changing only database.hostname and snapshot.mode):

{
  "name": "pg17-target-connector",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "database.hostname": "<TARGET_ENDPOINT>",
    "database.port": "5432",
    "database.user": "debezium_user",
    "database.password": "<PASSWORD>",
    "database.dbname": "testdb",
    "topic.prefix": "myapp",
    "schema.include.list": "test_cdc",
    "slot.name": "debezium_slot",
    "plugin.name": "pgoutput",
    "publication.name": "debezium_pub",
    "snapshot.mode": "never",
    "slot.drop.on.stop": "false",
    "heartbeat.interval.ms": "10000",
    "tombstones.on.delete": "true"
  }
}

Important: Keep the same topic.prefix value as your source connector. This allows Kafka consumers to continue reading from the same topics without reconfiguration.

3. Create the publication on the target cluster (Debezium needs a publication to filter WAL events):

-- On target:
CREATE PUBLICATION debezium_pub FOR TABLE
    test_cdc.orders,
    test_cdc.customers,
    test_cdc.inventory;

4. Notify downstream consumer teams of the upcoming pause in event delivery. In our testing the database-side cutover took about 2 seconds. The total consumer-visible pause also includes the time to start the connector on the target, which depends on your Kafka Connect setup.

To execute the cutover

This sequence pauses application writes briefly while the subscription drains and the Debezium slot is created on the target. We measured this cutover in a test environment. The setup used Aurora PostgreSQL 14.15 upgraded to 17.4, on db.t4g.medium instances in us-east-1. The workload was 65,000 rows across three tables under approximately 100 transactions per second (TPS) of sustained write load. Under those conditions, the database-side cutover completed in 2.21 seconds. Replication lag remained at 0 bytes during steady-state synchronization, and the initial data copy completed in seconds. Your actual duration depends on database size, write throughput, and network latency between source and target clusters.

The cutover duration is governed by how quickly the final replication lag drains to zero, which is a function of the in-flight transaction volume at cutover rather than total database size, plus the fixed steps (disable the subscription, synchronize sequences, and create the slot). Because the bulk data is already synchronized during the bridge phase, database size affects the synchronization phase, not the cutover window. Very high write rates or long-running transactions at cutover extend the lag-drain step.

Critical: To prevent duplicate events, pause the source connector and let it commit its offsets before you start the target connector. Don’t let both connectors stream at the same time. If both run simultaneously, even briefly, duplicate events occur because both process the same LSN range.

1. Stop application writes on the source cluster:

REVOKE INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA test_cdc FROM app_user;
SELECT NOW() AS write_stop_time, pg_current_wal_lsn() AS lsn_at_stop;

Production consideration: The REVOKE approach works for single-user applications. For production systems with connection poolers or multiple database users:

• Use application-level feature flags to stop writes at the service layer.
• Or set default_transaction_read_only = on in the parameter group (requires brief connection drain).
• Or issue a PAUSE command on your connection pooler (such as PgBouncer) to freeze all connections.

The REVOKE method shown here is suitable for testing and single-tenant applications.

2. Wait for the subscription lag to reach zero, then disable the subscription and wait for the apply worker to drain:

-- Poll until lag = 0:
SELECT pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn) AS lag_bytes
FROM pg_replication_slots
WHERE slot_name LIKE 'pg14_to_pg17%';

-- Disable the subscription on the target:
ALTER SUBSCRIPTION pg14_to_pg17_sub DISABLE;

-- Verify the apply worker has stopped:
SELECT pid FROM pg_stat_subscription
WHERE subname = 'pg14_to_pg17_sub';
-- Returns no rows

3. Synchronize sequences on the target. Logical replication doesn’t replicate sequence values, so you must reset them to avoid primary key conflicts:

-- On target: sync sequences using MAX values from REPLICATED data on the target.
-- This runs AFTER subscription is disabled and drained (Steps 1-2), confirming
-- no further inserts arrive. The MAX() queries the target table which already
-- has all replicated rows.
DO $$
DECLARE
    r RECORD;
    max_val BIGINT;
BEGIN
    FOR r IN
        SELECT s.sequencename, c.relname AS tablename, a.attname
        FROM pg_sequences s
        JOIN pg_depend d ON d.objid = (quote_ident(s.schemaname) || '.' || quote_ident(s.sequencename))::regclass
        JOIN pg_class c ON c.oid = d.refobjid
        JOIN pg_attribute a ON a.attrelid = c.oid AND a.attnum = d.refobjsubid
        WHERE s.schemaname = 'test_cdc'
    LOOP
        EXECUTE format('SELECT COALESCE(MAX(%I), 0) FROM test_cdc.%I', r.attname, r.tablename) INTO max_val;
        EXECUTE format('SELECT setval(''test_cdc.%I'', GREATEST(%s, 1))', r.sequencename, max_val);
        RAISE NOTICE 'Synced sequence test_cdc.% to %', r.sequencename, max_val;
    END LOOP;
END $$;

4. Drop the subscription cleanly and create the Debezium replication slot on the target:

-- Detach the slot before dropping (avoids dropping the source-side slot):
ALTER SUBSCRIPTION pg14_to_pg17_sub SET (slot_name = NONE);
DROP SUBSCRIPTION pg14_to_pg17_sub;

-- Create the Debezium slot on the target:
SELECT pg_create_logical_replication_slot('debezium_slot', 'pgoutput');

Validation gate: Before proceeding to Step 5, verify that the source connector is healthy and has consumed up to the current source LSN. Pausing it then commits a complete set of offsets to Kafka Connect’s __connect_offsets topic. Check the connector status:

curl -s http://localhost:8083/connectors/pg14-source-connector/status | jq '.tasks[0].state'
# Returns "RUNNING" at this point (you pause it in Step 5)

5. Stop the existing Debezium connector on the source and start the new connector pointed at the target:

# Stop the source connector:
curl -X PUT http://localhost:8083/connectors/pg14-source-connector/pause

# Deploy the new connector (using the config from the preparation step):
curl -X POST http://localhost:8083/connectors \
  -H "Content-Type: application/json" \
  -d @pg17-target-connector.json

Rollback procedure

If the cutover fails before Step 5 (connector swap), roll back to the source:

1. Re-enable the subscription on the target to maintain data synchronization:

ALTER SUBSCRIPTION pg14_to_pg17_sub ENABLE;

2. Restore write access on the source:

GRANT INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA test_cdc TO app_user;

3. Verify the source Debezium connector resumes normal operation by checking flush_lag returns to your baseline.

Important: Rollback is only safe if you have not yet started the target connector (Step 5). After the target connector begins writing to Kafka with the same topic.prefix, rolling back risks duplicate events in downstream consumers.

To validate the cutover

1. Verify the Debezium slot is active on the target:

SELECT slot_name, active, restart_lsn, confirmed_flush_lsn
FROM pg_replication_slots
WHERE slot_name = 'debezium_slot';
-- active returns 'true'

After the target connector attaches, the slot shows active = t:

   slot_name   | active | restart_lsn | confirmed_flush_lsn
---------------+--------+-------------+---------------------
 debezium_slot | t      | 0/4A130A18  | 0/4A130A18

2. Resume application writes on the target and verify events flow to Kafka:

-- Grant write access on target:
GRANT INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA test_cdc TO app_user;

-- Insert a verification record:
INSERT INTO test_cdc.orders (customer_id, product_name, amount, status)
VALUES (99999, 'CUTOVER_VERIFIED', 999.99, 'test');

Verify the event appears in your Kafka topic:

kafka-console-consumer.sh --bootstrap-server <KAFKA_ENDPOINT> \
  --topic myapp.test_cdc.orders \
  --max-messages 1

3. Monitor flush_lag on the target to verify the connector is consuming steadily:

SELECT application_name, state,
       sent_lsn, flush_lsn,
       flush_lag
FROM pg_stat_replication
WHERE application_name = 'debezium_slot';
-- flush_lag stays under your heartbeat interval (10s default)

To verify end-to-end continuity in our own validation, we inserted 100 marker rows on the target after the cutover. We then consumed the topic from the beginning. The 100 events appeared in the Kafka topic with distinct primary keys and no duplicates. This showed that the new connector resumed delivery without gaps or replays.

Handling failure scenarios

This section describes common failure scenarios for this pattern, with detection and resolution guidance for each. We reproduced the large-message (Scenario 3) and DDL-during-synchronization (Scenario 4) scenarios directly during testing. The remaining items describe expected PostgreSQL and Aurora behavior that you should validate in your own environment before relying on it in production.

Scenario 1: Aurora writer failover during synchronization

Symptoms: The subscription on the target stops receiving updates. pg_stat_subscription.last_msg_receipt_time becomes stale.

Detection: Monitor last_msg_receipt_time on the target. If it exceeds 60 seconds without update, the subscription may have lost its connection.

Resolution: An Aurora writer failover triggers an automatic DNS endpoint update. The subscription reconnects automatically once the new writer becomes reachable. Data loss isn’t expected to occur under normal circumstances, because the subscription is designed to resume from its last confirmed LSN. However, validate the reconnection time and behavior in your own environment, because it depends on your DNS caching and failover settings.

Prevention: Consider raising wal_sender_timeout on the source (for example, to 120 seconds) during the migration window to avoid premature sender shutdown across a failover transition. Avoid disabling it entirely (0) outside the migration window, because that also suppresses detection of genuinely dead connections.

Scenario 2: Debezium connector stall (network interruption)

Symptoms: Connector reports RUNNING in Kafka Connect, but flush_lag on the database grows without bound.

Detection: Query pg_stat_replication.flush_lag and alert if it exceeds 5 minutes. Don’t rely solely on the connector RUNNING state, which can mask this failure.

Resolution: Brief network interruptions generally recover on their own once connectivity returns. For a sustained stall, delete and recreate the connector. We observed during testing that a recreated connector resumes cleanly from the slot’s confirmed_flush_lsn with no data loss.

Prevention: Configure heartbeat.interval.ms (10000 or lower) and monitor flush_lag with automated alerting.

Scenario 3: Large messages exceed Kafka size limits

Debezium serializes each changed row into a single Kafka record, so a single wide row can exceed your Kafka producer or broker size limits. In testing, a row whose payload approached 1 MB failed to produce under the default max.request.size of 1,048,576 bytes (1 MB): rows up to roughly 1 MB of content delivered normally, while a row of approximately 1.05 MB produced a RecordTooLargeException. The Debezium JSON envelope (operation, source metadata, before and after images) adds overhead on top of the row data, so the serialized record crosses the limit slightly before the raw column data reaches it.

Symptoms: The connector task transitions to FAILED. The Kafka Connect log shows RecordTooLargeException. The connector doesn’t advance past the oversized record, so its committed offset stops moving and the database replication slot retains and accumulates WAL.

Detection: Connector task state shows FAILED with a RecordTooLargeException in the trace. The replication slot’s confirmed_flush_lsn stops advancing while pg_current_wal_lsn() continues to move, so retained WAL grows.

Resolution: Increase max.message.bytes at the broker and topic level and max.request.size in the connector’s producer overrides to accommodate the row, then restart the connector. In testing, the previously stuck record was delivered immediately on restart and the slot resumed draining. The connector cannot skip the oversized event, so the limits must be raised rather than worked around.

Prevention: Before cutover, audit JSONB, TEXT, and bytea columns that could approach 1 MB when serialized, and size max.message.bytes and max.request.size with headroom above your largest expected row.

Scenario 4: DDL changes applied during synchronization

Symptoms: After a DDL change on the source (for example, ALTER TABLE ADD COLUMN), replication for all tables in the publication stops. The subscription worker enters an error state.

Detection: pg_stat_subscription.last_msg_receipt_time stops advancing. Subscription worker logs report a schema mismatch.

Resolution: Apply the same DDL on the target manually, then re-enable the subscription. Logical replication does not replicate DDL statements.

Prevention: Freeze schema changes during the synchronization window. If DDL is unavoidable, apply it on the target first, then on the source.

Scenario 5: Long-running transactions block lag drain

Symptoms: After stopping application writes (Step 1 of cutover), replication lag remains non-zero for more than 30 seconds. The lag_bytes value doesn’t drain to zero because an open transaction on the source is holding back the replication stream.

Detection:

-- On source: identify blocking transactions
SELECT pid, usename, state, xact_start, query
FROM pg_stat_activity
WHERE state = 'idle in transaction'
  AND xact_start < NOW() - INTERVAL '30 seconds';

Resolution: Identify the blocking session and, if it is safe to do so, stop it with pg_terminate_backend. This forcibly terminates the backend and rolls back its in-flight transaction, so confirm with the application owner that the session can be safely ended before you run it:

-- Review the blocking session first:
SELECT pid, usename, state, query_start, left(query, 80) AS query
FROM pg_stat_activity
WHERE pid = <blocking_pid>;

-- Then terminate it if safe:
SELECT pg_terminate_backend(<blocking_pid>);

After the session ends, lag drains to zero within seconds and you can proceed with Step 2.

Prevention: Before initiating Step 1 (stopping writes), audit pg_stat_activity for long-running or idle-in-transaction sessions. Resolve them proactively. The subscription lag = 0 is a reliable drain signal only when no open transactions remain on the source.

Post-migration monitoring

After completing the cutover, establish ongoing monitoring for these three signals:

AuroraReplicaLag and flush_lag growth

Monitor the AuroraReplicaLag Amazon CloudWatch metric alongside pg_stat_replication.flush_lag for comprehensive visibility into replication health:

SELECT application_name,
       flush_lag,
       sent_lsn,
       flush_lsn,
       pg_wal_lsn_diff(sent_lsn, flush_lsn) AS pending_bytes
FROM pg_stat_replication
WHERE slot_name = 'debezium_slot';

Alert if flush_lag exceeds your heartbeat interval (default 10 seconds) for more than 5 minutes. A connector can report RUNNING while flush_lag grows for days, causing silent WAL accumulation on Aurora storage. As a projection, at an assumed 1 MB/s write rate, 24 hours of this failure would accumulate approximately 86 GB of unreclaimed WAL. Because flush_lag is not a native CloudWatch metric, publish it as a custom metric (for example, from a scheduled AWS Lambda function that queries pg_stat_replication) and create a CloudWatch alarm on it. For VolumeBytesUsed, alarm directly on the metric that Amazon RDS publishes.

WAL growth rate

Monitor the CloudWatch metric VolumeBytesUsed for the target cluster. Alert if the rate of growth exceeds your measured application write rate, which indicates the replication slot is retaining WAL faster than it is being consumed.

Kafka consumer offset divergence

Compare the Debezium connector’s latest committed offset with the Kafka topic’s high watermark. Growing divergence indicates the connector is writing to Kafka more slowly than events arrive, a sign of downstream backpressure or connector degradation.

Important: The Aurora CloudWatch metric AuroraReplicaLag monitors writer-to-reader replica lag, not subscription lag. You need custom monitoring (for example, an AWS Lambda function querying pg_stat_replication) for subscription-side visibility.

Automation at scale

For organizations managing dozens of clusters, you can encapsulate this pattern in an AWS Cloud Development Kit (AWS CDK) construct. The core automation components are:

• Cluster provisioning: CDK stack that creates the target cluster with matching parameter group, security groups, and subnet configuration.
• Schema replication: A Lambda function that runs pg_dump --schema-only and applies the output to the target.
• Cutover orchestrator: An AWS Step Functions state machine that executes the cutover sequence (disable subscription → drain → sync sequences → drop → create slot → swap connector) with gate checks between each step.
• Monitoring stack: CloudWatch alarms for AuroraReplicaLag, flush_lag, VolumeBytesUsed rate, and connector status.

Each cluster migration runs as an independent state machine execution. The Step Functions workflow provides built-in retry logic, timeout handling, and rollback triggers if any gate check fails.

Note: Multi-connector orchestration (4–8 connectors per database, as is common in large deployments) has not been validated in this pattern. If you operate multiple Debezium connectors per cluster, run a staging test in your environment before applying to production.

The SQL statements, connector configuration, and cutover sequence in this post are provided inline so you can adapt them to your environment. This walkthrough doesn’t include a separate code repository. The automation components described here are a reference design rather than a packaged construct.

Clean up

After verifying 24 hours of stable operation on the target cluster:

1. Delete the old Debezium connector pointed at the source:

curl -X DELETE http://localhost:8083/connectors/pg14-source-connector

2. Drop the Debezium replication slot on the source (stops WAL retention):

-- On source:
SELECT pg_drop_replication_slot('debezium_slot');

3. Drop the publication on the source:

DROP PUBLICATION debezium_pub;

4. After the target has run stably in production for a validation window (we suggest at least several days, for example 7–14 days, so you can roll back if needed), decommission the source cluster. Delete the DB instances in the cluster first, then delete the cluster:

# Delete the instance(s) first:
aws rds delete-db-instance \
  --db-instance-identifier pg14-source-instance \
  --skip-final-snapshot

# Wait for the instance to finish deleting, then delete the cluster:
aws rds delete-db-cluster \
  --db-cluster-identifier pg14-source \
  --skip-final-snapshot

Important caveats

Keep these limitations in mind when planning your migration:

• PostgreSQL 17 failover slots aren’t available on Aurora PostgreSQL. The community PostgreSQL 17 feature sync_replication_slots is not exposed as a modifiable parameter on Aurora. After a writer failover on Aurora PostgreSQL 17, replication slots must still be recreated and Debezium reconnected, the same operational behavior as PostgreSQL 14.
• TOAST columns with partial updates. When Debezium processes an UPDATE that doesn’t modify a TOASTed column (large TEXT or JSONB), it emits __debezium_unavailable_value as a placeholder. Verify your downstream consumers handle this sentinel value correctly.
• FOR ALL TABLES IN SCHEMA requires PostgreSQL 15 or later. On PostgreSQL 14, you must enumerate tables explicitly in your publication. Generate the table list dynamically from information_schema.tables.
• Sequences are not replicated. PostgreSQL logical replication does not synchronize sequence values. The setval() step during cutover is mandatory to prevent primary key conflicts when writes resume on the target.
• Silent WAL bloat risk. A Debezium connector can report RUNNING state while its flush_lag grows indefinitely. Monitor pg_stat_replication.flush_lag directly. Don’t rely on connector state alone.

Conclusion

In this post, we showed how you can migrate an Amazon Aurora PostgreSQL cluster across major versions while your Debezium CDC connectors keep streaming. You use native PostgreSQL logical replication to bridge the source and target clusters, run a brief cutover sequence that disables the subscription, synchronizes sequences, and creates a Debezium replication slot on the target, then redirect the connector with snapshot.mode=never. Because the connector keeps the same topic.prefix, your downstream consumers continue reading the same Kafka topics through the upgrade. We also covered the failure scenarios to plan for and the post-cutover monitoring that detects them.

PostgreSQL native logical replication provides a lightweight bridge for upgrading Aurora PostgreSQL major versions while preserving active Debezium CDC continuity. In our testing, the database-side cutover completed in roughly 2 seconds on a saturated db.t4g.medium test instance under load. Your duration will depend on instance size, write throughput, and data volume. In that same validation environment, we observed 100 out of 100 events delivered to Kafka with zero duplicates.

The approach uses two Aurora clusters (source and target) with no additional AWS services required for the data path. Every step in the cutover sequence is a SQL statement or API call, making it suitable for automation through Step Functions and CDK.

Before applying this pattern to production, validate it in a non-production environment with your actual schema, write volume, and Debezium connector count. Pay particular attention to flush_lag monitoring post-cutover, sequence synchronization for all serial and identity columns, and downstream consumer behavior during the brief event delivery pause.

About the authors