When I run MSCK REPAIR TABLE, Amazon Athena returns a list of partitions, but then fails to add the partitions to the table in the AWS Glue Data Catalog.

Some common causes of this behavior are:

  • The AWS Identity and Access Management (IAM) user or role doesn't have a policy that allows the glue:BatchCreatePartition action.
  • The Amazon Simple Storage Service (Amazon S3) path is in camel case instead of flat case (for example, userId instead of userid).

Allow glue:BatchCreatePartition in the IAM policy

Review the IAM policies attached to the user or role that you're using to execute MSCK REPAIR TABLE. When you use the AWS Glue Data Catalog with Athena, the IAM policy must allow the glue:BatchCreatePartition action. If the policy doesn't, then Athena can't add partitions to the metastore. For an example of an IAM policy that allows the glue:BatchCreatePartition action, see AmazonAthenaFullAccess Managed Policy.

Change the S3 path to flat case

The S3 path name must be in flat case. If the S3 path is in camel case, MSCK REPAIR TABLE doesn't add the partitions to the AWS Glue Data Catalog. For example, if your S3 path is userId, the following partitions aren't added to the AWS Glue Data Catalog:

s3://bucket/path/userId=1/

s3://bucket/path/userId=2/

s3://bucket/path/userId=3/

To resolve this issue, use flat case instead of camel case:

s3://bucket/path/userid=1/

s3://bucket/path/userid=2/

s3://bucket/path/userid=3/

Did this page help you? Yes | No

Back to the AWS Support Knowledge Center

Need help? Visit the AWS Support Center

Published: 2018-12-18