How can I set up cross-account access for EMRFS?

Last updated: 2020-01-17

I want to use the EMR File System (EMRFS) to write to Amazon Simple Storage Service (Amazon S3) buckets that are in a different AWS account.

Short Description

Use one of the following options to set up cross-account access for EMRFS:

  • Add a bucket policy for the destination bucket that grants access to the Amazon EMR account. This is easiest option. However, the destination account doesn't own the objects that EMRFS writes to the destination bucket.
  • Use a custom credentials provider. This option allows you to assume an AWS Identity and Access Management (IAM) role in the destination bucket account. This means that the destination account owns objects that EMRFS writes to the destination bucket.
  • Use role mappings in a security configuration. This option also allows EMRFS to assume an IAM role in the destination bucket account. This is the method that's discussed in this article.

Resolution

When you use a security configuration to specify IAM roles for EMRFS, you set up role mappings. A role mapping specifies an IAM role that corresponds to an identifier. In this scenario, the identifier is the Amazon S3 prefix that you want to access with EMRFS. Identifiers determine the basis for access to Amazon S3 through EMRFS. For more information, see Configure IAM Roles for EMRFS Requests to Amazon S3.

To create cross-account access for EMRFS using a security configuration with role mapping, follow these steps:

1.    Create an IAM role in the destination account. This is the role that you will assume from the EMR cluster.

2.    Add a trust policy similar to the following. The trust policy must allow the Amazon Elastic Compute Cloud (Amazon EC2) role for EMR to assume the role that you created in step 1. For more information, see Configure Roles.

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Principal":{
            "AWS":"arn:aws:iam::EMRFSAcctID:role/EMR_EC2_DefaultRole"
         },
         "Action":"sts:AssumeRole"
      }
   ]
}

3.    Use the AWS Command Line Interface (AWS CLI) to create a security configuration with a role mapping. The role mapping must specify the role in the destination account (the role that you created in step 1).

Note: You must use the AWS CLI or an SDK to create the security configuration. The console doesn't list roles in other accounts, even if you have permissions to assume those roles.

Supply a JSON object similar to the following for the role mapping. Replace these values in the example:
arn:aws:iam::DestinationAcctID:role/role_in_destination_account: the Amazon Resource Name (ARN) of the role that you created in step 1
s3://awsexamplebucket/: the bucket that you want EMRFS to write to.

{
  "AuthorizationConfiguration": {
    "EmrFsConfiguration": {
      "RoleMappings": [{
        "Role": "arn:aws:iam::DestinationAcctID:role/role_in_destination_account",
        "IdentifierType": "Prefix",
        "Identifiers": ["s3://awsexamplebucket/"]
      }]
    }
  }
}

4.    Create an IAM policy and then attach it to the EMR EC2 instance profile (for example, EMR_EC2_DefaultRole).

The following example policy allows AWS Security Token Service (STS) to assume all roles. At a minimum, your policy must allow STS to assume the role that you created in step 1. For more information, see Granting Permissions to Create Temporary Security Credentials.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "*"
        }
    ]
}

5.    Launch an EMR cluster and specify the security configuration that you created in step 3.

Note: If the destination bucket uses server-side encryption with AWS Key Management Service (AWS KMS), the assumed role must be a key user in the KMS customer master key (CMK). You can't access the bucket if the role isn't listed in the KMS CMK.


Did this article help you?

Anything we could improve?


Need more help?