How can I troubleshoot access denied or unauthorized operation errors with an IAM policy?

Last updated: 2020-11-03

My AWS Identity and Access Management (IAM) policy returned an error or my IAM policy isn't working as expected. How can I troubleshoot issues with an IAM policy?

Short description

To troubleshoot issues with IAM policies:

  • Identify the API caller
  • Evaluate the IAM policy
  • IAM policy error message troubleshooting examples

Resolution

Understand the error messages and identify the API caller

Important:

Be sure that the API calls are made on behalf of the correct IAM identity before reviewing the IAM policy. If the error message doesn't include the caller information, follow these steps to identify the API caller:

  1. Open the AWS Management Console.
  2. In the upper right corner of the page, choose the arrow next to the account information.
  3. If you're signed in as an IAM role, choose the arrow next to the role name for the source credentials, account information, destination role name, and account information.
  4. If you're signed in as a federated user, your IAM role is listed under Federated Login.

You can use the AWS CLI command get-caller-identity to identify the API caller. You can also use the AWS CLI command with the --debug flag to identify the source of the credentials from the output similar to the following:

2018-03-13 16:23:57,570 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials

Evaluate the IAM policy

Verify if the proper permissions are granted to the API caller by checking the IAM policies attached. For more information, see Determining whether a request is allowed or denied within an account.

After you verify that your API caller has the proper IAM permissions, review the resource-level permissions. If the API actions don't support resource-level permissions, specify the wildcard * in the resource element of the IAM statement. You can also view the Policy summary to troubleshoot resource-level permission issues. To view the policy summary:

  1. Open the IAM console.
  2. In the navigation pane, choose Policies.
  3. Choose the arrow next to the policy name to expand the policy details view.
  4. Choose Policy summary.

In the following example, the policy only provides permission to StartInstances, but the policy doesn't grant permission to DeleteSubnet. Because the DeleteSubnet action doesn't support resource-level permission, you can't specify the Amazon Resource Name (ARN) in the resource element. Use an asterisk instead: "Resource": "*"

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PermissionNotGrantedAsAPIDoesNotSupportResourceLevelPermission",
            "Effect": "Allow",
            "Action": "ec2:DeleteSubnet",
            "Resource": "arn:aws:ec2:us-east-1:123456789012:subnet/subnet-254cf47c"
        },
        {
            "Sid": "PermissionGrantedAsAPISupportResourceLevelPermission",
            "Effect": "Allow",
            "Action": "ec2:StartInstances",
            "Resource": "arn:aws:ec2:us-east-1:123456789012:instance/i-0e58cfa95b858b1ce"
        }
    ]
}

After you have confirmed your resource-level permissions, review the conditions used in your IAM policies. AWS global condition context keys can be used with all AWS services that support IAM for access control. Service specific condition keys can only be used within the specific service—EC2 conditions to EC2 API actions. For more information, see Actions, resources, and condition context keys for AWS services. If you use multiple conditions, see Multiple values in a condition.

In this example policy, the condition is matched if an IAM API request is called by IAM user admin and the source IP address is from 1.1.1.0/24 or 2.2.2.0/24.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "iam:*",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:username": "admin"
                },
                "IpAddress": {
                    "aws:SourceIp": [
                        "1.1.1.0/24",
                        "2.2.2.0/24"
                    ]
                }
            }
        }
    ]
}

Review any service-specific controls that may affect the permissions. For example, VPC endpoint policies and AWS Organizations Service Control Policies (SCP) can also affect the permission evaluation. For more information, see Controlling access to services with VPC endpoints.

IAM policy error message troubleshooting examples

See the following examples to identify the error message, the API caller, the API, and the resources being called:

 

Example error message API caller API Resources When
A: "An error occurred (UnauthorizedOperation) when calling the DeleteKeyPair operation: You are not authorized to perform this operation." unknown DeleteKeyPair unknown time of the error occurs
B: "An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:iam::123456789012:user/test is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::123456789012:role/EC2-FullAccess" arn:aws:iam::123456789012:user/test AssumeRole arn:aws:iam::123456789012:role/EC2-FullAccess time of the error occurs
C: "An error occurred (AccessDenied) when calling the GetSessionToken operation: Cannot call GetSessionToken with session credentials" unknown GetSessionToken unknown time of the error occurs
D: "An error occurred (UnauthorizedOperation) when calling the AssociateIamInstanceProfile operation: You are not authorized to perform this operation. Encoded authorization failure message: ...." unknown AssociateIamInstanceProfile unknown time of the error occurs

 

Using this evaluation method, you can identify the cause of the error messages you can receive for permission issues for different AWS services. For more details, see the following common error messages and troubleshooting steps:

Example error message A:

This error message indicates that you don't have permission to call the DeleteKeyPair API.

  1. Identify the API caller.
  2. Confirm that the ec2:DeleteKeyPair API action isn't included in any deny statements.
  3. Confirm that the ec2:DeleteKeyPair API action is included in the allow statements.
  4. Confirm that there's no resource specified for this API action. Note: This API action doesn't support resource-level permissions.
  5. Confirm that all IAM conditions specified in the allow statement are supported by the delete-key-pair action and that the conditions are matched.

Example error message B:

This error message includes the API name, API caller, and target resource. Be sure that the IAM identity that called the API has the correct access to the resources. Review the IAM policies using the previous evaluation method and evaluate the following:

The trust policy of IAM role: EC2-FullAccess:

  1. Confirm arn:aws:iam::123456789012:user/test or arn:aws:iam::123456789012:root isn't included in any deny statement of the trust policy.
  2. Confirm arn:aws:iam::123456789012:user/test or arn:aws:iam::123456789012:root is included in the allow statement of the trust policy.
  3. Confirm all IAM conditions specified in that allow statement are supported by sts:AssumeRole API action and matched.

The IAM policies attached to the API caller (arn:aws:iam::123456789012:user/test):

  1. Confirm arn:aws:iam::123456789012:role/EC2-FullAccess isn't included in any deny statement with sts:AssumeRole API action.
  2. If arn:aws:iam::123456789012:root is in the allow statement of the trust policy, confirm arn:aws:iam::123456789012:role/EC2-FullAccess is included in the allow statement of the IAM policies with sts:AssumeRole API action.
  3. Confirm all IAM conditions specified in that allow statement are supported by sts:AssumeRole API action and match.

Example error message C:

This error message indicates that get-session-token isn't supported by temporary credentials. For more information, see Comparing the AWS STS API operations.

Example error message D:

This error message returns an encoded message that can provide details about the authorization failure. To decode the error message and get the details of the permission failure, see DecodeAuthorizationMessage. After decoding the error message, identify the API caller and review the resource-level permissions and conditions.

Review the IAM policy permissions:

  1. If the error message indicates that the API is explicitly denied, remove ec2:AssociateIamInstanceProfile or iam:PassRole API actions from the matched statement.
  2. Confirm that ec2:AssociateIamInstanceProfile and iam:PassRole are in the allow statement with supported and correct resource targets. For example, confirm that the resource targets of ec2:AssociateIamInstanceProfile API action are EC2 instances and the resource targets of iam:PassRole are IAM roles.
  3. If ec2:AssociateIamInstanceProfile and iam:PassRole API actions are in the same allow statement, confirm that all conditions are supported by ec2:AssociateIamInstanceProfile and iam:PassRole API action and that the conditions match.
  4. If ec2:AssociateIamInstanceProfile and iam:PassRole API actions are in separate allow statements, confirm that all conditions in each allow statement are supported by an action and that the conditions match.