How do I resolve problems with my Amazon RDS MySQL DB instance that's using more storage than expected?
Last updated: 2019-11-14
My Amazon Relational Database Service (Amazon RDS) database instance on MySQL is using more space than expected. Why is this happening, and how can I optimize disk storage?
You can use the FreeStorageSpace Amazon CloudWatch metric to monitor the available storage space for an RDS DB instance, but that metric doesn't describe how the DB instance is consuming storage.
Use the following strategies to reclaim storage space:
Run OPTIMIZE TABLE
Some space consumed by tables isn't actively in use, but is allocated to the table anyway. If innodb_file_per_table is enabled (it's enabled by default), you can reclaim that space using OPTIMIZE TABLE. OPTIMIZE TABLE works for InnoDB, MyISAM, and ARCHIVE tables, and while Amazon RDS accepts the OPTIMIZE TABLE command, Amazon RDS executes ALTER TABLE...FORCE. When this occurs, you can receive a warning message similar to the following: "Table does not support optimize, doing recreate + analyze instead." For more information, see the MySQL documentation for OPTIMIZE TABLE.
To check for fragmentation, run a query similar to the following:
SELECT table_name, data_length, max_data_length, index_length, data_free FROM information_schema.tables WHERE table_schema='schema_name' ;
The data_free column highlights the amount of free space allocated to a table that isn't actively in use. You might be able to reclaim this space with OPTIMIZE TABLE if the table is created in a separate tablespace according to the Amazon RDS default innodb_file_per_table configuration setting. For more information, see the MySQL documentation for File-Per-Table Tablespaces.
Reduce application table storage
To see how much storage is used by application tables on your DB instance, run a query similar to the following:
SELECT table_schema, SUM(data_length + index_length + data_free)/1024/1024 AS total_mb, SUM(data_length)/1024/1024 AS data_mb, SUM(index_length)/1024/1024 AS index_mb, SUM(data_free)/1024/1024 AS free_mb, COUNT(*) AS tables, CURDATE() AS today FROM information_schema.tables GROUP BY table_schema ORDER BY 2 DESC ;
To locate the largest application table on your DB instance, run a query similar to the following:
SELECT table_schema, table_name, (data_length + index_length + data_free)/1024/1024 AS total_mb, (data_length)/1024/1024 AS data_mb, (index_length)/1024/1024 AS index_mb, (data_free)/1024/1024 AS free_mb, CURDATE() AS today FROM information_schema.tables ORDER BY 3 DESC ;
Note: The total storage used by an individual database and table can't be calculated if the database includes tables with variable-length columns that are longer than 768 bytes (for example, BLOB, TEXT, VARCHAR, or VARBINARY).
Reduce binary log storage
Adding a read replica causes the master instance's binary log to use additional storage. To find out how much storage the binary log on the master instance is using, check the BinLogDiskUsage CloudWatch metric. A large amount of growth can indicate that one or more Read Replicas are not synchronized. For more information, see Accessing MySQL Binary Logs.
Reduce or disable general log and slow query log storage
Enabling the general log and slow query log parameters causes your DB instance to begin storing these logs and the backups of these logs. To rotate these files and control disk usage, see mysql.rds_rotate_general_log and mysql.rds_rotate_slow_log.
Note: To avoid potential performance and disk use issues, disable the general and slow query logs when you're not actively using them to troubleshoot.
Manage or reduce InnoDB system tablespace size
The system tablespace contains the InnoDB data dictionary and undo space, and it starts out at 10 MB. After space is allocated, the file will always be at least that size, though long-running transactions can consume more available storage.
By default, Amazon RDS sets innodb_file_per_table to 1, which means that data for each tablespace is stored in its own .ibd file. To recover space that is marked as reusable for related tables, use the OPTIMIZE TABLE command to resize per table tablespace files, or drop a table.
If innodb_file_per_table is set to 0, all tables are also allocated to the system tablespace. Dropping tables or indexes, or deleting or truncating data from tables allocated in the system tablespace marks the space previously occupied as reusable. However, that command doesn't free up any space to the file system.
Because it's not possible to shrink the system tablespace in-place, export your current database's data, and then import the data into a new instance. To reduce downtime, configure your new MySQL instance as a replica of the source Amazon RDS master instance. When the replica is synchronized with the source Amazon RDS master instance, switch to the new instance. For information about manual replication, see Replication with a MySQL or MariaDB Instance Running External to Amazon RDS.
Note: Restoring from a snapshot or creating a Read Replica won't help you recover space from the system tablespace. This is because both methods use a snapshot of the source instance storage volume that contains the system tablespace.