How can I configure IAM task roles in Amazon ECS to avoid "Access Denied" errors?

Last updated: 2019-07-16

I receive an "Access Denied" error message when my application makes AWS API calls. How can I resolve this error?

Short Description

If you don't configure IAM task roles correctly, you can receive "Access Denied" error messages when your application makes AWS API calls.

To avoid this error, provide your AWS Identity and Access Management (IAM) task role in the task definition for Amazon Elastic Container Service (Amazon ECS). Your tasks can use this IAM role for AWS API calls. The IAM task role must have all the permissions required by your application. If a task can't find the IAM task role due to configuration issues, then the Amazon Elastic Compute Cloud (Amazon EC2) instance role is used instead.

Resolution

To correctly configure IAM roles for your task, check the following:

Confirm that the ECS container agent is running

To confirm that the ECS container agent is running, run the following command:

docker ps

Enable IAM roles in your ECS container agent configuration file

1.    Open your /etc/ecs/ecs.config file.

2.    To enable IAM roles for tasks in containers with bridge and default network modes, set ECS_ENABLE_TASK_IAM_ROLE to true. See the following example:

ECS_ENABLE_TASK_IAM_ROLE=true

3.    To enable IAM roles for tasks in containers with the host network mode, set ECS_ENABLE_TASK_IAM_ROLE_NETWORK_HOST to true. See the following example:

ECS_ENABLE_TASK_IAM_ROLE_NETWORK_HOST=true

4.    To update the configuration file, restart the AECS container agent by running either of the following commands:

For Amazon ECS-optimized Amazon Linux AMIs:

sudo stop ecs
sudo start ecs

For Amazon ECS-optimized Amazon Linux 2 AMIs:

sudo systemctl restart ecs

Confirm that your IAM policy has the correct trust relationship with your Amazon ECS tasks

To confirm that the IAM role has the correct trust relationship, update your IAM policy as follows:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Verify proxy settings for the ECS container agent

If you're using HTTP_PROXY on your Amazon ECS container agent configuration, apply the NO_PROXY setting as follows:

NO_PROXY=169.254.169.254,169.254.170.2,/var/run/docker.sock

Confirm that you're using the right AWS SDK

The application running in your container must use a version of the AWS SDK no older than the July 2016 version.

To update your AWS SDK, see Tools to Build on AWS.

Meet the requirements for non-Amazon ECS Optimized AMIs

If you're using a non-Amazon ECS Optimized AMI, set the required rules for iptables.

Note: If you restart the instance, the rules for iptables are reset to the default. To avoid a reset, run one of the following commands to save the rules:

For Amazon ECS-optimized Amazon Linux AMIs:

sudo service iptables save

For Amazon ECS-optimized Amazon Linux 2 AMIs:

sudo iptables-save | sudo tee /etc/sysconfig/iptables && sudo systemctl enable --now iptables

Make the credential path environment variable available to non-PID 1 processes

The environment variable AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is available only to PID 1 processes within a container. If the container is running multiple processes or init processes (such as wrapper script, start script, or supervisord), the environment variable is unavailable to non-PID 1 processes.

To set your environment variable so that it's available to non-PID 1 processes, export the environment variable in the .profile file. For example, run the following command to export the variable in the Dockerfile for your container image:

RUN echo 'export $(strings /proc/1/environ | grep AWS_CONTAINER_CREDENTIALS_RELATIVE_URI)' >> /root/.profile

Now additional processes can access the environment variable.

Note: There's a dependency on the strings and grep commands when you export the environment variable.


Did this article help you?

Anything we could improve?


Need more help?