Get started with Amazon Elasticsearch Service: Use Amazon Cognito for Kibana access control

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details.

Welcome to this introductory series on Elasticsearch and Amazon Elasticsearch Service (Amazon ES). In this and future blog posts, we provide the basic information that you need to get started with Elasticsearch on AWS.

Introduction

On April 2, 2018, we released integration between Amazon Elasticsearch Service and Amazon Cognito. You can now provide and manage user-level sign-on for Kibana access to your Amazon ES domains. With Amazon Cognito, you can connect to external identity providers to provide single sign-on for your users. You can also set access policies for users or groups of users, making it easy to manage access control.

This post walks you through the setup for adding Amazon Cognito authentication and access control to Kibana in Amazon ES domains. In the first half, you create the basic components, including an Amazon ES domain, an Amazon Cognito user pool, and an Amazon Cognito identity pool. At the end of this process, you sign in to Kibana based on the shared Auth_Role provided through Amazon Cognito. The Auth_Role is shared by all users in the Amazon Cognito user pool.

In the second half of the post, you create an IAM role and a policy attached to that role that you assign to a specific user. You can generalize the methodology to provide differential access to your Amazon ES domains.

Be sure to have a look at this prior blog post, How to Control Access to Your Amazon Elasticsearch Service Domain, which covers setting IAM policies for your Amazon ES domains. In a future post, we will cover connecting Amazon Cognito to an external identity provider. For more detailed documentation, see the Amazon ES user guide and the Amazon Cognito user guide.

Part one: Create the resources

Start by creating the basic components that you’ll need to complete the process, including an Amazon Cognito user pool and identity pool, and an Amazon ES domain.

Set up Amazon Cognito

First, you create the Amazon Cognito components that provide identity and roles for controlling access to Amazon ES. The two main components of Amazon Cognito are user pools and identity pools. User pools are scalable user directories that enable you to provide sign-up and sign-in for your app users. Identity pools provide temporary, limited-privilege, AWS credentials to users after they sign in. When you use Amazon Cognito with Amazon ES, you create a user pool, and Amazon ES connects it with your domain. You import or create identities in the identity pool and manage the policies connected with those identities.

First, let’s create an Amazon Cognito user pool. Open the AWS Management Console and sign in. Type Cognito in the search box, and choose Cognito in the drop-down list. On the main splash page, choose Manage your User Pools. Then choose Create a user pool.

Give the user pool a name. In this example, I named my pool kibana_access. You can step through the settings or choose the defaults. For now, choose Review defaults.

Scroll down, and choose Create pool.

Before you can integrate with Amazon ES, you need to create an Amazon Cognito domain name. This name can be the same or different from your Amazon ES domain name. It provides Amazon Cognito with DNS to support the authentication UI.

If you’re not already looking at your user pool’s details, choose your user pool to view the details, and then choose Domain name in the navigation pane. Type a name in the text box. In this example, I used kibana-auth-domain. Be sure to choose Save changes.

Now you create an identity pool. At the top of the console, choose Federated Identities.

If you haven’t created any identity pools yet, you’ll drop into a wizard for creating a new identity pool. Give your identity pool a name. I chose kibana_identities. Then select the Enable access to unauthenticated identities check box.

Choose Create Pool. You are redirected to the AWS Identity and Access Management (IAM) console to create roles for unauthenticated and authenticated users. If you choose View Details, you see that two role policies were created for you: Cognito_<identity pool name>Auth_Role and Cognito_<identity pool name>UnAuth_Role. By default, when you sign in to Kibana, you assume the Auth_Role. Unauthenticated users assume the UnAuth_role. In part two of this post, you change that behavior to add access controls for individual users and groups. Choose Allow.

Create an Amazon ES domain

You have completed the preliminaries and can now create an Amazon ES domain. The basic steps are described here. For a more detailed walkthrough, see the Amazon ES documentation.

Open the Amazon ES console. Create a new domain and give it a name. I used kibana-with-auth. Use the latest Elasticsearch version—currently, version 6.2, and choose Next.

On the configuration page, leave the default settings for Instance count, Instance type, Enable dedicated master, Enable zone awareness, and all the options under Storage configuration, Snapshot configuration, and Advanced options. Choose Next.

On the next page, you set up your access controls, including Amazon Cognito. For this post, we use a domain that has public access. Choose the Public access option. Select the check box to Enable Amazon Cognito for authentication. This reveals settings for Amazon Cognito. For Cognito User Pool, choose kibana_access. For Cognito Identity Pool, choose kibana_identities. Leave the default settings for IAM Role Name and Role Policy.

You must also set an access policy for the domain. For Set the domain access policy to, choose Allow or deny access to one or more AWS accounts or IAM users. Copy your AWS account ID and paste it into the Account ID or ARN box. Choose OK, and then choose Next on the main page. You will set up access to the domain in Amazon Cognito, but you need to set a policy on the Amazon ES domain to create it. Review the configuration, and choose Confirm.

When you confirm the changes, as part of the domain creation, Amazon ES links your Amazon Cognito user pool and identity pool. You authorize Amazon ES to perform that action through the CognitoAccessForAmazonES role specified earlier.

Wait for your domain to become Active. It takes about 10 minutes. You can move forward when your dashboard shows endpoints for your domain.

Note: Be sure to copy and save your domain’s ARN (Amazon Resource Name). You will need it in a minute.

Create a user and a group

Now you need to add a user group and a user, and modify the IAM policy to sign in to Kibana. Navigate to the Amazon Cognito console, and choose Manage your User Pools. Choose kibana_access. In the navigation pane, choose Users and groups. Then choose Create user.

In the resulting dialog box, type a Username, Temporary password, and Email. Clear the Mark phone number as verified? check box.

Choose Create user. You return to the Users and groups page, where you should see your new user.

Choose the Groups tab, and then choose Create group.

In the resulting dialog box, type a Name and a Description. Leave the IAM role blank, and set the Precedence to 0. Then choose Create group.

At this point, you can sign in to Kibana. You will assume the Amazon Cognito Auth_Role as described earlier. However you haven’t set sufficient permissions to actually load Kibana all the way. You can change the domain’s policy to provide access for the Auth_Role. Navigate to the Amazon ES console, choose your domain, and choose Modify access policy.

Change the Principal to the ARN for the assumed Auth role.

"Principal": {
    "AWS": "arn:aws:sts::<your account id>:assumed-role/Cognito_<your identity pool name>Auth_Role/CognitoIdentityCredentials"
}

Then choose Submit. Wait for the Domain Status to become Active. Choose the Kibana URL to open it.

You should see a sign-in dialog box like the following. Sign in as kibana_user.

You must change your password (the default behavior for the Amazon Cognito user pool). If all has gone well, you will see the Kibana splash page. Note that the service adds a Logout button at the bottom of the page.

Choose Logout. You can then customize the access as described in the next section.

Part two: Customize access

Now that you have the basic components set up, you’re ready to create an IAM role for the group of Amazon Cognito users and attach it to the group.

Create an IAM policy and role

Navigate to the IAM console. Choose Policies in the left navigation pane, and then choose Create policy.

On the resulting page, choose the JSON tab, and replace the contents with the following policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "es:esHttp*"
      ],
      "Resource": "arn:aws:es:region:123456789012:domain/domain-name/*"
    }
  ]
}

Note: es:esHttp* allows access to all Elasticsearch REST verbs. You can further limit access by setting, e.g., es:esHttpGet as the sole action the user may perform. This will effectively grant read-only access to the domain.

Replace the domain ARN in the Resource in this example with your domain’s ARN.

Important! When you copy and paste the domain ARN, be sure to retain or add the trailing “/*” in the policy.

Choose Review policy. On the following page, give your policy a Name and a Description (I chose kibana-with-auth_open_policy). Choose Create policy.

This policy allows Kibana to access your domain. You can customize it to assign different permissions for different Kibana users. At present, the service supports domain-level access control through the Resource in the policy. It does not support dashboard or visualization access control for Kibana dashboards. However, Actions that you allow or deny will apply in the Kibana developer tools tab. For example, you can add restrictions denying es:ESHttpDelete to prevent Kibana users from sending DELETE requests.

Now, choose Roles in the left navigation pane. Choose Create role. Under Choose the service that will use this role, choose EC2. You won’t be using Amazon EC2, but you need to choose a service, and EC2 is a placeholder. Choose Next:Permissions.

On the Attach permissions policies page, in the Filter box, type the name of the policy that you just created. Be sure to place a check mark in the box to the left of the policy in the list. Then choose Next:Review.Give the role a Name and a Description. In this example, I chose kibana-with-auth_role as my role name. Verify that your policy is listed in the Policies. Choose Create role.

You can create different roles for different types of access and attach them to different user groups in Amazon Cognito. Doing this allows you to restrict access to different Amazon ES domains for different groups in your organization.

You also need to allow Amazon Cognito to assume the role that you just created. At this point, you should be looking at your roles in the IAM console. If you’re not, navigate to the IAM console, and choose Roles in the left navigation pane. Type your role’s name in the Filter box, and then choose the role to edit it. Choose the Trust relationships tab, and then choose Edit trust relationship.

Replace the contents of the Edit Trust Relationship Policy Document with the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "cognito-identity.amazonaws.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity"
    }
  ]
}

Choose Update Trust Policy.

Wire everything together

Now that you’ve created a role for the group of Amazon Cognito users, you can attach it to the group. Navigate to the Amazon Cognito console. Make sure that you choose User Pools, and then choose the kibana_access user pool to edit it.

Choose Users and groups in the left navigation pane. Choose the Groups tab, and then choose kibana_group. In the top section, choose the Edit icon to edit the group. In the Update Group dialog box, expand the list and choose the IAM role that you just created. You might have to refresh the page if the IAM role does not appear in the drop-down list.

Choose Update group. Then choose Add users in the second section of the Groups, kibana_group page. Choose the plus sign next to kibana_user to add the user to the group.

Close the dialog box.

The last thing you need to do is configure the identity pool to use the role and policy from the user pool instead of the default, Cognito_<user_pool>Auth_Role and Cognito_<user_pool>Unauth_Role. If you don’t do this step, all authenticated users will assume the Auth_Role and all unauthenticated users will assume the Unauth_role.

At the top of the page, choose Federated Identities to view your identity pools. Choose your identity pool (kibana_identities) to edit. In the upper-right corner of the page, choose Edit identity pool. Scroll down and choose the down arrow to reveal the Authentication providers settings. Under Authenticated role selection, open the drop-down list and select Choose role from token.

Amazon Cognito will now use the role that you attached to your group to authenticate. Choose Save Changes.

Sign in to Kibana

You have set up an Amazon Cognito user pool, an identity pool, and an Amazon ES domain that uses Amazon Cognito for signing in to Kibana. Time to try it out! Navigate to the Amazon ES dashboard. Choose your domain, locate the Kibana URL, and choose the link to open it.

You can now sign in as kibana_user and operate with access provided through the kibana-with-auth_role.

Conclusion

Congratulations! You’ve created an Amazon Cognito user pool and identity pool with a custom role that you attached to an Amazon Cognito user. You configured an Amazon Elasticsearch Service domain to use Amazon Cognito to allow your users to sign in to Kibana and scoped the access of those users to your domain. You can add more users and groups for other domains and control access to your Amazon ES deployments throughout your organization.

About the Author

Jon Handler (@_searchgeek) is an AWS solutions architect specializing in search technologies. He works with our customers to provide guidance and technical assistance on database projects, helping them improve the value of their solutions when using AWS.

AWS Database Blog