How do I resolve problems with my Amazon RDS MySQL DB instance that is using more storage than expected?

Last updated: 2018-12-28

My Amazon Relational Database Service (Amazon RDS) database instance on MySQL is using more space than expected. Why is this happening, and how can I optimize disk storage?

Resolution

You can use the FreeStorageSpace Amazon CloudWatch metric to monitor the available storage space for an RDS DB instance, but that metric doesn't describe how the DB instance is consuming storage.

Use the following strategies to reclaim storage space:

Run OPTIMIZE TABLE

Some space consumed by tables isn't actively in use, but is allocated to the table anyway. If innodb_file_per_table is enabled (it is enabled by default), you can reclaim that space using OPTIMIZE TABLE. For more information, see the MySQL documentation for OPTIMIZE TABLE

To check for fragmentation, run a query similar to the following:

SELECT
	table_name,
	data_length,
	max_data_length,
	index_length,
	data_free
FROM
	information_schema.tables 
WHERE table_schema='schema_name'
;

The data_free column highlights the amount of free space allocated to a table that isn't actively in use. You might be able to reclaim this space with OPTIMIZE TABLE if the table is created in a separate tablespace as per the Amazon RDS default innodb_file_per_table configuration setting. For more information, see the MySQL documentation for innodb_file_per_table.

Reduce application table storage

To see how much storage is used by application tables on your DB instance, run a query similar to the following:

SELECT 
	table_schema,
	SUM(data_length + index_length + data_free)/1024/1024 AS total_mb,
	SUM(data_length)/1024/1024 AS data_mb,
	SUM(index_length)/1024/1024 AS index_mb,
	SUM(data_free)/1024/1024 AS free_mb,
	COUNT(*) AS tables,
	CURDATE() AS today 
FROM 
	information_schema.tables
	GROUP BY table_schema
	ORDER BY 2 DESC
;

To locate the largest application table on your DB instance, run a query similar to the following:

SELECT 
	table_schema,
	table_name,
	(data_length + index_length + data_free)/1024/1024 AS total_mb,
	(data_length)/1024/1024 AS data_mb,
	(index_length)/1024/1024 AS index_mb,
	(data_free)/1024/1024 AS free_mb,
	CURDATE() AS today
FROM 
	information_schema.tables
	ORDER BY 3 DESC
;

Note: The total storage used by an individual database and table can't be calculated if the database includes tables with variable-length columns that are longer than 768 bytes (for example, BLOB, TEXT, VARCHAR, or VARBINARY).

Reduce binary log storage

Adding a read replica causes the master instance's binary log to use additional storage. To find out how much storage the binary log on the master instance is using, check the BinLogDiskUsage CloudWatch metric. A large amount of growth can indicate that one or more Read Replicas are not synchronized. For more information, see Accessing MySQL Binary Logs.

Reduce or disable general log and slow query log storage

Enabling the general log and slow query log parameters causes your DB instance to begin storing these logs and the backups of these logs. To rotate these files and control disk usage, see mysql.rds_rotate_general_log and mysql.rds_rotate_slow_log.

Note: To avoid potential performance and disk use issues, disable the general and slow query logs when you're not actively using them to troubleshoot.

Manage or reduce InnoDB system tablespace size

The system tablespace contains the InnoDB data dictionary and undo space, and it starts out at 10 MB. After space is allocated, the file will always be at least that size, though long-running transactions can consume more available storage.

By default, Amazon RDS sets innodb_file_per_table to 1, which means that data for each tablespace is stored in its own .ibd file. To recover space that is marked as reusable for related tables, use the OPTIMIZE TABLE command to resize per table tablespace files, or drop a table.

If innodb_file_per_table is set to 0, all tables are also allocated to the system tablespace. Dropping tables or indexes, or deleting or truncating data from tables allocated in the system tablespace marks the space previously occupied as reusable. However, that command doesn't free up any space to the file system.

Because it's not possible to shrink the system tablespace in-place, export your current database's data and then import the data into a new instance. To reduce downtime, configure your new MySQL instance as a subordinate of the source Amazon RDS master instance. When the subordinate is synchronized with the source Amazon RDS master instance, switch to the new instance. For information about manual replication, see Replication with a MySQL or MariaDB Instance Running External to Amazon RDS.

Note: Restoring from a snapshot or creating a Read Replica won't help you recover space from the system tablespace. This is because both methods use a snapshot of the source instance storage volume that contains the system tablespace.