How can I resolve the "DBInstance xxxxxx failed to stabilize" error when I create an RDS DB instance from snapshots using AWS CloudFormation?

3 minute read
0

When I try to create an AWS::RDS::DBInstance resource with AWS CloudFormation, I get the following error: "DBInstance xxxxxx failed to stabilize."

Short description

If you create an Amazon Relational Database Service (Amazon RDS) DB instance (AWS::RDS::DBInstance) from an existing RDS DB snapshot using AWS CloudFormation, then you can receive an error due to the following reasons:

  • The storage type in the DB snapshot doesn't match the storage type in the DB instance.
  • Your DB snapshots contain invalid objects.

Note: There are other possible causes of RDS DB instance stabilization issues. The steps in the following resolution apply only to issues where the RDS DB instance is created from snapshots in AWS CloudFormation.

Resolution

Match the storage types between your DB snapshot and DB instance

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

1.    Verify the storage type of the DB snapshot by using the Amazon RDS console or the describe-db-snapshots AWS CLI commands. For example:

aws rds describe-db-snapshots --db-snapshot-identifier MyRDSSnapShot --region us-east-1
{
    "DBSnapshots": [
        {
            .......
            "AllocatedStorage": 20, 
            "Status": "available", 
            "PercentProgress": 100, 
            "DBSnapshotArn": "arn:aws:rds:us-east-1:1234567890:snapshot:MyRDSSnapShot", 
            "EngineVersion": "12.1.0.2.v7", 
            "ProcessorFeatures": [], 
            "OptionGroupName": "default:oracle-ee-12-1", 
            "SnapshotCreateTime": "2019-03-16T17:43:41Z", 
            "AvailabilityZone": "us-east-1f", 
            "StorageType": "gp2",
            .......
        }
    ]
}

Note: The StorageType value gp2 from the AWS CLI command maps to the value General Purpose (SSD) in the Amazon RDS console.

2.    In the AWS::RDS::DBInstance resource definition of your AWS CloudFormation JSON or YAML template, set the value of StorageType to match the storage type of the snapshot. In this case, the storage type is set to gp2.

JSON:

{
    "MySampleDB": {
        "Type": "AWS::RDS::DBInstance",
        "Properties": {
            "DBInstanceClass": "db.t2.small",
            "StorageType": "gp2",
            "Engine": "postgres",
            "EngineVersion": "9.5.12",
            ........
        }
    }
}

YAML:

MySampleDB:
  Type: 'AWS::RDS::DBInstance'
  Properties:
    DBInstanceClass: db.t2.small
    StorageType: gp2
    Engine: postgres
    EngineVersion: 9.5.12
    ......

Note: If the storage type isn't set, then AWS CloudFormation sets StorageType to standard by default, which might not match the storage type of the referenced snapshot. If the storage types don't match, then RDS could take longer to create the database because of the extra storage conversion process. Also, this mismatch could lead to timeout errors for larger databases. It's a best practice to explicitly define the storage type in the AWS::RDS::DBInstance resource. Also, consider setting the storage type to match the storage type of the snapshot.

3.    Launch your AWS CloudFormation stack to create the DB instance.

Verify that your DB snapshots don't have invalid objects

If the DB snapshot that's used to create your DB instance has invalid objects, complete the following steps:

1.    Identify and fix the invalid objects in your existing databases.

For example, the following Oracle database query returns a list of invalid objects:

select owner,object_name,object_type,status from dba_objects where status='INVALID'

2.    Retake snapshots from your existing databases.

Important: Retake the DB snapshots only after you fixed or removed all the invalid objects from your databases.

3.    Launch your AWS CloudFormation stack to create the DB instance using the existing DB snapshot.


AWS OFFICIAL
AWS OFFICIALUpdated 3 years ago