How to develop a user-facing data application with IAM Identity Center and S3 Access Grants (Part 1)

This is Part 1 of a two-part blog series: Configuring the application. Here is Part 2: Developing the application.

When we at AWS talk to our customers about their data lakes, they usually describe a desired access pattern in which users and groups from a corporate directory are granted access to datasets in Amazon Simple Storage Service (Amazon S3), usually organized by prefix or “folder.”

It has long been possible to implement this permission scheme using AWS Identity and Access Management (IAM) techniques. The implementation pattern usually looked like the below, in which:

The application authenticates the user, typically via a single sign-on redirect flow to the identity provider’s sign-in page.
And then, to access data on behalf of the authenticated user, the application either:
1. Accesses data under a single IAM role for the application, making decisions in business logic on which users should see which data, or
2. Maps different users to IAM roles with respectively different permissions, with business logic choosing the right IAM role for the right user.

Figure 1: Two patterns for mapping permissions of users and groups to data: The pattern on the left implements this mapping directly in the application’s business logic. The pattern on the right has the application mapping groups to IAM roles.

Either of these IAM-focused approaches is effective. However, both raise a challenge for larger or more complex data lakes: the mapping of users and groups to data or IAM roles is in the application’s business logic. Thus, when it comes time to add a second application to the lake, this business logic must be re-implemented, creating the need for mechanisms to validate that the two implementations are consistent. And because the permissions scheme is implemented in the application, away from the data, they won’t be visible to AWS services such as Spark on EMR that might also be part of the data lake. Finally, because the application must map the user into an IAM role before accessing the data, the CloudTrail data events for the S3 accesses do not reflect the original user’s identity, which adds undifferentiated work to the task of auditing who (which person) accessed which data.

AWS recently announced two new features — S3 Access Grants and IAM Identity Center Trusted Identity Propagation — that, together, can be used to implement these common users/groups-to-data permissions schemes centrally. S3 Access Grants is a new feature of S3 that allows customers to grant access, directly and within S3, for directory users and groups or IAM principals, to prefixes in S3. Trusted Identity Propagation is a new mechanism in AWS’s IAM Identity Center (successor to AWS Single Sign-On) by which these direct references to users and groups are possible; it enables AWS services to support user identities directly.

Rather than maintaining and managing its own mapping of users and groups to data with business logic, a user-facing data application that is integrated with S3 Access Grants has the simpler task of requesting just-in-time access to the data in S3, on behalf of the signed-in user.

Figure 2: Data application built on S3 Access Grants with Trusted Identity Propagation. Permission mappings are defined directly in S3 Access Grants.

S3 Access Grants gives S3 customers a single, consistent place to define the mappings of users and groups to data, removing the need to implement this logic inside the data application.

This two-part blog post is intended for application developers. We will demonstrate how to build a typical data application that authenticates its users and then connects those authenticated users directly to data in S3, using S3 Access Grants and Trusted Identity Propagation. Our detailed walkthrough will feature a reference application we built that authenticates its users with Microsoft Entra ID, formerly known as Azure Active Directory. Similar implementations are possible with Okta or any other identity provider that can be integrated with IAM Identity Center and offers user authentication functionality for applications.

In this first of two parts, we will cover the details of how to configure this data application, both in Microsoft Entra ID as well as in IAM Identity Center. In Part 2, we will show you excerpts of the application’s code as it integrates with both IAM Identity Center and S3 Access Grants to authenticate an end-user and then access data on their behalf.

About IAM Identity Center

IAM Identity Center, as the name suggests, is the hub for all things in AWS that have to do with workforce end-user identities from a corporate identity provider such as Microsoft Entra ID. Since its launch, which was originally under the name AWS Single Sign-On, it has provided single sign-on, or federation, to the AWS Management Console for cloud operators in an organization. The new feature we’re discussing in this blog post, Trusted Identity Propagation, expands IAM Identity Center’s capabilities to support direct references to identities within AWS services. It is this new capability that enables S3 Access Grants to grant access to directory users and groups directly.

Thus, in order to use identities with AWS services, a prerequisite is to configure synchronization (SCIM) from your identity provider to IAM Identity Center. Here are AWS’s setup instructions, and Microsoft Entra ID documentation on the process.

A note on use of AWS accounts in this blog post

As a best practice, you enable a single instance of IAM Identity Center for your AWS Organization. However, it is also possible to create an IAM Identity Center instance that is local to an individual AWS account, which can be useful in test environments or for proofs of concept. Throughout this walkthrough, we will be assuming that your AWS Organization has a single IAM Identity Center instance, which is the recommended configuration. We will call the account that contains the IAM Identity Center instance 222222222222, and it is not the same AWS account as the one in which you would typically deploy your data application, which we will call account 111111111111. A setup like that will be typical in production environments, in which the people who operate the Identity Center instance will often be distinct from the people who develop individual workloads. In fact, the configurations described here would work (with appropriate IAM permissions in place) even if your S3 buckets and S3 Access Grants instance were in yet a third account, which would be the case if your S3 data were owned by a different group of people. If, instead, you are using an account instance of IAM Identity Center, for example to build a proof of concept, you will need the application and the S3 buckets to be in that same AWS account as well.

Application overview

The application in this blog post is an “Enterprise application” in Microsoft Entra ID. That means it has the ability to authenticate end-users by redirecting them to Entra ID’s sign-in page. It will make use of Trusted Identity Propagation and S3 Access Grants to access data in S3, on behalf of the currently signed-in user, for example to display a visualization of the data.

Figure 3: Summary of user-facing flow in our data application. Users log in and then request a view of the data through our application.

At a high level, the user’s interaction with data in your application might work like this:

Log in with Entra ID. The end-user visits your application and, through a normal single-sign-on interaction, authenticates to Microsoft Entra ID.
Token exchange with IAM Identity Center. Your application will exchange the identity token from Microsoft with IAM Identity Center, for an identity token usable within AWS.
User interaction with your application. Your application presents its users with a means of interacting with data. For example, it might provide a visualization of data requested by the user (for example, a pie chart) that will require it to read objects from S3 on the user’s behalf.
Request access to data from S3 Access Grants. Your application will request access to the necessary data in S3 to fulfill the user’s request.
Read S3 objects. Your application will read the data from S3 and display an appropriate result (for example a pie chart) to the end user.

In the second part of this two-part series, we will show you how to implement each of the above steps in your application. In this first part, we will show you how to configure your data application.

Configuring your data application for use with IAM Identity Center’s Trusted Identity Propagation

The actions described here are one-time actions. The goal is to configure an entity called an “Application” in IAM Identity Center to represent the application you are building. Specifically, in this step, you will give Identity Center all the metadata it needs – both on your Entra ID tenant as well as your application – to be able to trust the identity tokens by which your application will represent its authenticated users.

You will take the following application configuration steps:

A. Create a Trusted Token Issuer in IAM Identity Center. This construct will represent your Microsoft Entra ID tenant within IAM Identity Center, enabling it to recognize identity tokens for your application’s authenticated users.

B. Create an Application in IAM Identity Center to represent your application as it interacts with IAM Identity Center on behalf of signed-in users.

C. Connect your Application to the Trusted Token Issuer so that your application will able to present Entra ID-issued identity tokens for authenticated users to IAM Identity Center.

D. Connect your Application to S3 Access Grants so that it can make requests to S3 Access Grants on behalf of authenticated users.

E. Associate your Application in Identity Center with the AWS account in which it will run. This step will allow your application to make the necessary requests to its corresponding IAM Identity Center Application.

We will describe each of these configuration steps in detail.

Step A: Setting up the Trusted Token Issuer relationship between Entra ID and IAM Identity Center

The first thing you need to do is to tell IAM Identity Center that your Entra ID tenant will be issuing the original identity tokens for its authenticated users as they log in. This relationship is called a Trusted Token Issuer. With the configuration of this Trusted Token Issuer, IAM Identity Center will be able to verify the tokens’ signatures and accept them as valid tokens of user identity.

The Trusted Token Issuer concept in IAM Identity Center can be used to enable a variety of patterns, beyond the specific use case we will focus on here.

Notice that this step is not specific to any one specific data application. As you develop more enterprise applications in your Entra ID tenant that you wish to integrate with IAM Identity Center, you will not need to repeat this step; rather, you can reuse this Trusted Token Issuer.

The account that owns the instance of IAM Identity Center in your AWS Organization needs to take this step. It can be done either in the IAM Identity Center console or programmatically. In the IAM Identity Center console, find the “Create trusted token issuer” button on the Settings page, under the Authentication tab.

Figure 4: IAM Identity Center console. Trusted Token Issuers can be found on the Authentication tab of the Settings page.

Here is the metadata that you will configure for the Trusted Token corresponding to your Entra ID tenant.

Figure 5: Configuring a Trusted Token Issuer corresponding to our Entra ID tenant

Explaining the relevant fields above:

The IssuerUrl is the value that IAM Identity Center should expect in the iss field of Microsoft’s identity tokens. The 7139b665-… GUID is your Entra ID tenant identifier (assigned by Microsoft). By convention, the public keys for verifying the tokens’ signatures will be at /.well-known/openid-configuration relative to the IssuerUrl. Behind the scenes, IAM Identity Center will retrieve those keys from there and use them to validate tokens. (In the API, you will specify the JwksRetrievalOption as OPEN_ID_DISCOVERY, which means exactly that.)
Identity provider attribute (called ClaimAttributePath in the API) and IAM Identity Center attribute (called IdentityStoreAttributePath in the API) tell IAM Identity Center how, given an incoming identity token from Microsoft for a specific user, to map this user to its corresponding, synchronized user in IAM Identity Center. Specifically, the Microsoft-issued tokens will put a GUID that uniquely identifies the user in the oid field, whereas in our IAM Identity Center directory, that GUID will be in the externalId field.

Step B: Configuring our application in IAM Identity Center

Because your application is an Enterprise Application in Microsoft Entra ID, it will be registered there.

Figure 6: Enterprise application registered in Microsoft Entra ID

The important field here is the Application ID, 46600a38-…. When users sign into the application, Entra ID will send them an identity token with this value in the aud field. That indicates that this token is supposed to be used by this specific application.

You also need to register this same application as an AWS-side entity, in IAM Identity Center. This API request, as well as the following steps to configure the application, can be made from the AWS account into which you deploy application (111111111111 in these examples), even though that is likely in a different AWS account from the one containing your IAM Identity Center instance (222222222222). It’s only if you are using an account instance of IAM Identity Center that everything needs to be in the same AWS account.

If you intend for everyone in the directory to be able to use your application, you will indicate that “assignments” (i.e., designations of which users and groups can use the application) are not required.

> aws sso-admin create-application \
    --instance-arn "arn:aws:sso:::instance/ssoins-12345678" \
    --application-provider-arn "arn:aws:sso::aws:applicationProvider/custom" \
    --name MyDataApplication 
{
    "ApplicationArn": "arn:aws:sso::222222222222:application/ssoins-12345678/apl-87654321"
}

> aws sso-admin put-application-assignment-configuration \
    --application-arn arn:aws:sso::222222222222:application/ssoins-12345678/apl-87654321 \
    --no-assignment-required

In the next step, you will associate this application with the Trusted Token Issuer that you created to represent your Entra ID tenant.

Step C: Connecting the application to the Trusted Token Issuer representing an Entra ID tenant

In this step, you will configure IAM Identity Center to expect the identity tokens that your application is issued by Entra ID as users authenticate. The Trusted Token Issuer, which you had set up in Step A, represents your Entra ID tenant as a known issuer of tokens. Now, you will associate it with your IAM Identity Center application here. IAM Identity Center calls this association an “application grant.”

> cat /tmp/grant.json
{
  "JwtBearer": {
    "AuthorizedTokenIssuers": [
      {
        "TrustedTokenIssuerArn": "arn:aws:sso::222222222222:trustedTokenIssuer/ssoins-12345678/tti-01234567-…", \
        "AuthorizedAudiences": ["46600a38-…"]
      }
    ]
  }
}

> aws sso-admin put-application-grant \
    --application-arn "arn:aws:sso::222222222222:application/ssoins-12345678/apl-87654321" \
    --grant-type "urn:ietf:params:oauth:grant-type:jwt-bearer" \
    --grant file:///tmp/grant.json

This request to PutApplicationGrant tells IAM Identity Center that your application will present identity tokens from your Microsoft Entra ID tenant, as represented by the Trusted Token Issuer, and that the audience for this token will be your Microsoft-assigned enterprise application id, that is the GUID 46600a38-… We will see that all in action in Part 2 of this series, as we discuss how to develop the application.

Step D: Connecting the application to S3 Access Grants

The next step is to tell IAM Identity Center that your application intends to make requests to S3 Access Grants on behalf of its authenticated users. This relationship is called a “scope” in IAM Identity Center: It has to do with the scope of actions that can be taken by this application on behalf of authenticated users. In your application’s case, the desired scope is the ability to make API requests to S3 Access Grants. We’ll see that in action in Part 2 as well.

> aws sso-admin put-application-access-scope \
    --application-arn "arn:aws:sso::222222222222:application/ssoins-12345678/apl-87654321" \
    --scope "s3:access_grants:read_write"

Step E: Cross-account permissions for the IAM Identity Center Application resource

And there’s just one more step you need to take to configure your IAM Identity Center Application: You need to tell IAM Identity Center that it will get API calls – namely requests to exchange Entra ID tokens from Microsoft for IAM Identity Center tokens for onward use within AWS – from an IAM principal that represents your application in AWS. These API interactions with IAM Identity Center will be normal, signed AWS-authenticated and -authorized API requests, and they will be cross-account. In terms of IAM authorization, the Principal (an IAM role associated with your deployed application) will be in account 111111111111, and the Resource (the IAM Identity Center Application) will be in account 222222222222. An IAM resource-based policy will be needed to allow this API call to be authorized. You will set that up here, as an “application authentication method.” It is a resource-based policy on the Application resource in IAM Identity Center. (Even if the application is deployed in the same AWS account as the IAM Identity Center instance, this resource-based policy needs to be in place.)

Of course, that means that your application, which will run in AWS account 111111111111, will be running under some IAM role in that AWS account. We’ll talk more about that IAM role in Part 2; we’ll call it the Application IAM role. It’s likely to be the IAM role associated with the compute environment on which the application runs. For example, if your application runs on an AWS Lambda function, your Application IAM role will be the execution IAM role associated with that Lambda function.

> cat /tmp/authentication-method.json
{
  "Iam": {
    "ActorPolicy": {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Principal": { 
            "AWS": "111111111111" 
          },
          "Action": "sso-oauth:CreateTokenWithIAM",
          "Resource": "arn:aws:sso::222222222222:application/ssoins-12345678/apl-87654321",
          "Condition": {
            "ArnEquals": {
              "aws:PrincipalArn": "arn:aws:iam::111111111111:role/ApplicationRole"
            }
          }
        }
      ]
    }
  }
}

> aws sso-admin put-application-authentication-method \
--application-arn "arn:aws:sso::222222222222:application/ssoins-12345678/apl-87654321" \
--authentication-method file:///tmp/authentication-method.json \
--authentication-method-type IAM

You’ve just configured an Application in IAM Identity Center. To summarize the steps you took:

A. You created a Trusted Token Issuer, which represents your Microsoft Entra ID tenant in IAM Identity Center. The original user identity tokens will be issued by this tenant.

B. You created an Application resource in IAM Identity Center to represent your application as it interacts with IAM Identity Center on behalf of its authenticated users.

C. You connected your Application to your Trusted Token Issuer. This “grant” tells IAM Identity Center to expect this application to exchange its Entra ID tokens for the IAM Identity Center tokens that it will be able to use in AWS.

D. You connected our Application to S3 Access Grants. This “scope” tells IAM Identity Center that your application intends to interact with S3 Access Grants on behalf of its authenticated users.

E. You configured an IAM resource-based policy for your IAM Identity Center Application, allowing interactions from the IAM role that will represent your application. IAM Identity Center calls this an “authentication method”; it allows a CreateTokenWithIAM API call that will be at the center of your application’s interactions with IAM Identity Center in Part 2.

S3 Access Grants: Granting access to a directory group

For this example application, we will assume that you have also configured S3 Access Grants with some grants. In Part 2, we will consider a user named Mateo Jackson, who is a member of the Red Project Group. This directory group has access to all data associated with projects concerning things that are colored red, for example tomatoes.

In order to use S3 Access Grants with IAM Identity Center identities, you will first need to associate the S3 Access Grants instance in the account and region with IAM Identity Center, which you can do either programmatically or in the S3 Access Grants console.

The grant allowing the Red Project Group to access the necessary prefix in S3 looks like this in S3 Access Grants. (There may be many other grants in S3 Access Grants, but this one is relevant to our particular walkthrough.) In this example, that directory group has the identifier in IAM Identity Center starting with 0374d892 that is shown below.

Not shown in this blog post, but when granting access to IAM Identity Center users and groups in S3 Access Grants, you can even limit your grant to a specific IAM Identity Center Application. As in, we could have allowed the Red Project Group access to this prefix in S3, but only when the users are accessing the data through your particular data application. By default, the grants are not specific to a particular IAM Identity Center Application.

For a more detailed walkthrough on how to configure grants in S3 Access Grants and how they work, please read Scaling data access with Amazon S3 Access Grants.

Figure 7: S3 Access Grant that grants read-only access to a directory group

Conclusion

At the end of Part 1 of this two-part blog post series, you have successfully configured an enterprise application in Microsoft Entra ID for use with the new Trusted Identity Propagation feature of IAM Identity Center. Having taken these steps, you are ready for Part 2, where we will give you the details of your data application’s integration, as it authenticates a user and then accesses data directly on that user’s behalf. See you soon!