How can I use an SSH tunnel to access Kibana from outside of a VPC with Amazon Cognito authentication?

Last updated: 2020-08-27

My Amazon Elasticsearch Service cluster is in a virtual private cloud (VPC). I want to use an SSH tunnel to access Kibana from outside the VPC with Amazon Cognito authentication.

Short description

By default, Amazon Cognito restricts Kibana access to AWS Identity and Access Management (IAM) users in the VPC. However, you can access Kibana from outside the VPC using an SSH tunnel.

Important: Be sure that when accessing Kibana (a third-party tool) from outside the VPC, it is compliant with your organization's security requirements.

To access Kibana from outside the VPC using an SSH tunnel, perform the following steps:

1.    Create an Amazon Cognito user pool and identity pool.

2.    Create an Amazon Elastic Compute Cloud (Amazon EC2) instance in a public subnet in the same VPC that the Elasticsearch domain is in.

3.    Use a browser add-on, such as FoxyProxy, to configure a SOCKS proxy.

4.    Create an SSH tunnel from your local machine to the EC2 instance.

Note: You can also use an NGINX proxy or Client VPN to access Kibana from outside a VPC with Amazon Cognito authentication. For more information, see How can I access Kibana from outside of a VPC using Amazon Cognito authentication?

5.    (Optional) If fine-grained access control (FGAC) is enabled, add an Amazon Cognito authenticated role.

Resolution

Create an Amazon Cognito user pool and identity pool

1.    Create an Amazon Cognito user pool.

2.    Configure a hosted user pool domain.

3.    In the Amazon Cognito console navigation pane, choose Users and groups.

4.    Choose Create user, and then complete the fields. Be sure to enter an email address. Then, select the Mark email as verified check box.

5.    Choose the Groups tab, and then choose Create group. For Precedence, enter 0. For more information, see Creating a new group in the AWS Management Console.

6.    Open the Amazon Cognito console again.

7.    Choose Manage Identity Pools, and then choose Create new identity pool.

8.    Enter a name for your identity pool, select the check box to Enable access to unauthenticated identities, and then choose Create Pool.

9.    When you are prompted for access to your AWS resources, choose Allow to create the two default roles associated with your identity pool—one for unauthenticated users and one for authenticated users.

10.    Configure your Amazon ES domain to use Amazon Cognito authentication for Kibana.
For Cognito User Pool, choose the user pool that you created in step 1.
For Cognito Identity Pool, choose the identity pool that you created in step 8.

11.    Configure your Amazon ES domain to use an access policy similar to the following. Replace these values:
account-id with your AWS account ID
identity-name with the name of your Amazon Cognito identity pool
ES-name with the name of your Amazon ES domain
Region with the Region that your Amazon ES domain is in, such as us-east-1

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:sts::account-id:assumed-role/Cognito_identity-nameAuth_Role/CognitoIdentityCredentials"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:Region:account-id:domain/ES-name/*"
    }
  ]
}

For example, the following access policy uses these values:

AWS account ID: 111122223333
Amazon Cognito identity pool name: MyIdentityPool
Amazon ES domain name: MyES
Region: us-east-1

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:sts::111122223333:assumed-role/Cognito_MyIdentityPoolAuth_Role/CognitoIdentityCredentials"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:111122223333:domain/MyES/*"
    }
  ]
}

Create an EC2 instance and configure security group rules

1.    Launch an EC2 instance in a public subnet of the VPC that your Elasticsearch domain is in. On the Configure Instance Details page, be sure that Auto-assign Public IP is set to Enable.
Note: In the following steps, the EC2 instance is referred to as tunnel_ec2.

2.    Add inbound rules to the security group associated with the tunnel_ec2 instance. These rules must allow traffic to ports 8157 and 22 from the IP address of the local machine that you'll use to access the Kibana dashboard.

3.    Add an inbound rule to the security group associated with the Elasticsearch domain. This rule must allow traffic from the private IP address of the tunnel_ec2 instance.

Configure the SOCKS proxy

1.     Add FoxyProxy Standard to Google Chrome.

2.    Open FoxyProxy, and then choose Options.

3.    In the Proxy mode drop-down list, choose Use proxies based on their pre-defined patterns and priorities.

4.    Choose Add New Proxy.

5.    Select the General tab and enter a Proxy Name, such as "Kibana Proxy."

6.    On the Proxy Details tab, be sure that Manual Proxy Configuration is selected and then complete the following fields:
For Host or IP Address, enter localhost.
For Port, enter 8157.
Select SOCKS proxy
Select SOCKS v5.

7.    Choose the URL Patterns tab.

8.    Choose Add new pattern and then complete the following fields:
For Pattern Name, enter a name that makes sense to you, such as "VPC Endpoint."
For URL pattern, enter the VPC endpoint for Kibana. Be sure that Whitelist URLs is selected. Be sure that Wildcards is selected.

9.     Choose Save.

Create the SSH tunnel

1.    Run the following command from the local machine that you'll use to access the Kibana dashboard. Replace the following:
mykeypair.pem: the name of the .pem file for the key pair that you specified when you launched the tunnel_ec2 EC2 instance.
public_dns_name: the public DNS of your tunnel_ec2 EC2 instance. For more information, see Viewing DNS Hostnames for Your EC2 Instance.

ssh -i "mykeypair.pem"  ec2-user@public_dns_name -ND 8157

2.    Enter the Kibana endpoint in your browser. The Amazon Cognito login page for Kibana appears.

(Optional) If fine-grained access control (FGAC) is enabled, add an Amazon Cognito authenticated role

If fine-grained access control (FGAC) is enabled on your Elasticsearch cluster, you might encounter a "missing role" error. To resolve the "missing role" error, perform the following steps:

1.    Sign in to your AWS Management Console.

2.    Under Analytics, choose Elasticsearch Service.

3.    Choose Actions.

4.    Choose Modify master user.

5.    Choose Set IAM ARN as your master user.

6.    In the IAM ARN field, add the Amazon Cognito authenticated ARN role.

7.    Choose Submit.

For more information about fine-grained access control, see Tutorial: IAM master user and Amazon Cognito.