Why does my IAM user have access to Data Catalog resources even after their permissions are restricted in Lake Formation?

Last updated: 2021-03-25

I restricted table permissions for a user in AWS Lake Formation (for example, SELECT with ExcludedColumnNames). However, the user can still query the table using Amazon Athena.

Short description

Lake Formation grants the Super permission to the IAMAllowedPrincipals group by default. This permission applies to all existing AWS Glue Data Catalog resources. If Use only IAM access control is enabled, then Lake Formation also grants the Super permission on new resources. This means that the AWS Identity and Access Management (IAM) policies control access to Data Catalog resources and Amazon Simple Storage Service (Amazon S3) locations. For more information, see About upgrading to the Lake Formation permissions model.

To restrict access, change the default settings to make sure that you're using the Lake Formation permissions model for new and existing Data Catalog resources.

Resolution

To update the security settings for all new databases and tables in your data lake, see Change Data Catalog settings.

To update the security settings only for certain new resources, create the resource, and then update the permissions only on that resource. For more information, see Secure existing Data Catalog resources.

The change in the default security settings applies only to the resources that are created after security settings are updated.

To change the security settings for existing databases, explicitly remove the permissions from IAMAllowedPrincipals on each database by doing the following:

  1. Open the AWS Lake Formation console.
  2. In the navigation pane, under Data catalog, choose Databases.
  3. Select the circle next to the database that you want to update.
  4. Choose Actions, and then choose Edit.
  5. Under Default permissions for newly created tables, clear Use only IAM access control for new tables in this database.
  6. Choose Save.
  7. On the Databases page, be sure that the database is selected. Choose Actions, and then choose Revoke.
  8. In the Revoke permissions dialog box, in the IAM users and roles list, scroll down to the Group heading, and then choose IAMAllowedPrincipals.
  9. Under Database permissions, be sure that Super is selected, and then choose Revoke.

To change the security settings for existing tables, explicitly remove the permissions from IAMAllowedPrincipals on each table by doing the following:

  1. Open the AWS Lake Formation console.
  2. In the navigation pane, choose Tables.
  3. On the Tables page, select the circle next to the table that you want to update.
  4. Choose Actions, and then choose Revoke.
  5. In the Revoke permissions dialog box, in the IAM users and roles list, scroll down to the Group heading, and then choose IAMAllowedPrincipals
  6. Under Table permissions, be sure that Super is selected, and then choose Revoke

Note: Be sure that the Amazon S3 path to the database or table is registered with Lake Formation.

After changing the security settings for existing resources, grant table permissions to the IAM user or role. You can do this using the Lake Formation console or the AWS Command Line Interface (AWS CLI).

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.

The following are examples of CLI commands that grant the SELECT permission on specific columns:

Allow SELECT on a specific column only:

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT"  --resource '{ "TableWithColumns": {"DatabaseName":"retail", "Name":"inventory", "ColumnNames": ["prodcode","location","period","withdrawals"]}}'

Exclude columns (allow SELECT on all columns except the column that's specified under ExcludedColumnNames):

aws lakeformation grant-permissions --principal DataLakePrincipalIdentifier=arn:aws:iam::111122223333:user/datalake_user1 --permissions "SELECT"  --resource '{ "TableWithColumns": {"DatabaseName":"retail", "Name":"inventory", "ColumnWildcard": {"ExcludedColumnNames": ["location"]}}}'

To perform Lake Formation operations, principals must have Lake Formation and IAM permissions. In addition to this to query the table in Athena user must also have permission to run Athena queries. For example, to query a table from Athena when Lake Formation is managing permissions, the user must have the following IAM permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "minimumAWSGlueLakeFormationAthenapermissiontorunselectqueryontable",
            "Effect": "Allow",
            "Action": [
                "lakeformation:GetDataAccess",
                "glue:SearchTables",
                "glue:GetTables",
                "glue:GetPartitions",
                "glue:GetDatabases",
                "glue:GetTable",
                "glue:GetDatabase",
                "athena:GetNamespace",
                "athena:ListWorkGroups",
                "athena:GetCatalogs",
                "athena:GetNamespaces",
                "athena:GetExecutionEngine",
                "athena:GetExecutionEngines",
                "athena:GetTables",
                "athena:GetTable"
            ],
            "Resource": "*"
        },
        {
            "Sid": "AthenaPermissionToRunQueryInAWorkergroup",
            "Effect": "Allow",
            "Action": [
                "athena:StartQueryExecution",
                "athena:GetQueryResults",
                "athena:DeleteNamedQuery",
                "athena:GetNamedQuery",
                "athena:ListQueryExecutions",
                "athena:StopQueryExecution",
                "athena:GetQueryResultsStream",
                "athena:ListNamedQueries",
                "athena:CreateNamedQuery",
                "athena:GetQueryExecution",
                "athena:BatchGetNamedQuery",
                "athena:BatchGetQueryExecution"
            ],
            "Resource": [
                "arn:aws:athena:*:*:workgroup/*"
            ]
        },
        {
            "Sid": "allowS3PermissionToSaveAthenaQueryResults",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucketMultipartUploads",
                "s3:AbortMultipartUpload",
                "s3:CreateBucket",
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": [
                "arn:aws:s3:::aws-athena-query-results-*"
            ]
        }
    ]
}

Did this article help?


Do you need billing or technical support?