How can I use an SSH tunnel to access Kibana from outside of a VPC with Amazon Cognito authentication?

Last updated: 2019-07-15

My Amazon Elasticsearch Service cluster is in a virtual private cloud (VPC). I want to use an SSH tunnel to access Kibana from outside the VPC with Amazon Cognito authentication.

Short Description

By default, Amazon Cognito restricts Kibana access to AWS Identity and Access Management (IAM) users in the VPC. To access Kibana from outside the VPC using an SSH tunnel:

1.    Create an Amazon Cognito user pool and identity pool.

2.    Create an Amazon Elastic Compute Cloud (Amazon EC2) instance in a public subnet in the same VPC that the Elasticsearch domain is in.

3.    Use a browser add-on, such as FoxyProxy, to configure a SOCKS proxy.

4.    Create an SSH tunnel from your local machine to the EC2 instance.

Note: You can also use an NGINX proxy or Client VPN to access Kibana from outside a VPC with Amazon Cognito authentication. For more information, see How can I access Kibana from outside of a VPC using Amazon Cognito authentication?

Resolution

Important: Your Elasticsearch domain is more secure when you restrict access to users in the VPC. Before you continue, be sure that this procedure does not violate your organization's security requirements.

Create an Amazon Cognito user pool and identity pool

1.    Create an Amazon Cognito user pool.

2.    Configure a hosted user pool domain.

3.    In the Amazon Cognito console navigation pane, choose Users and groups.

4.    Choose Create user, and then complete the fields. Be sure to enter an email address. Then, select the Mark email as verified check box.

5.    Choose the Groups tab, and then choose Create group. For Precedence, enter 0. For more information, see Creating a New Group in the AWS Management Console.

6.    Open the Amazon Cognito console again.

7.    Choose Manage Identity Pools, and then choose Create new identity pool.

8.    Enter a name for your identity pool, select the check box to Enable access to unauthenticated identities, and then choose Create Pool.

9.    When you are prompted for access to your AWS resources, choose Allow to create the two default roles associated with your identity pool—one for unauthenticated users and one for authenticated users.

10.   Configure your Amazon ES domain to use Amazon Cognito authentication for Kibana.
For Cognito User Pool, choose the user pool that you created in step 1.
For Cognito Identity Pool, choose the identity pool that you created in step 8.

11.   Configure your Amazon ES domain to use an access policy similar to the following. Replace these values:
account-id with your AWS account ID
identity-name with the name of your Amazon Cognito identity pool
ES-name with the name of your Amazon ES domain
Region with the Region that your Amazon ES domain is in, such as us-east-1

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:sts::account-id:assumed-role/Cognito_identity-nameAuth_Role/CognitoIdentityCredentials"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:Region:account-id:domain/ES-name/*"
    }
  ]
}

For example, the following access policy uses these values:

AWS account ID: 111122223333
Amazon Cognito identity pool name: MyIdentityPool
Amazon ES domain name: MyES
Region: us-east-1

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:sts::111122223333:assumed-role/Cognito_MyIdentityPoolAuth_Role/CognitoIdentityCredentials"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:111122223333:domain/MyES/*"
    }
  ]
}

Create an EC2 instance and configure security group rules

1.    Launch an EC2 instance in a public subnet of the VPC that your Elasticsearch domain is in. On the Configure Instance Details page, be sure that Auto-assign Public IP is set to Enable.
Note: In the following steps, the EC2 instance is referred to as tunnel_ec2.

2.    Add inbound rules to the security group associated with the tunnel_ec2 instance. These rules must allow traffic to ports 8157 and 22 from the IP address of the local machine that you'll use to access the Kibana dashboard.

3.    Add an inbound rule to the security group associated with the Elasticsearch domain. This rule must allow traffic from the private IP address of the tunnel_ec2 instance.

Configure the SOCKS proxy

1.     Add FoxyProxy Standard to Google Chrome.

2.    Open FoxyProxy, and then choose Options.

3.    In the Proxy mode drop-down list, choose Use proxies based on their pre-defined patterns and priorities.

4.    Choose Add New Proxy.

5.    Select the General tab and enter a Proxy Name, such as "Kibana Proxy."

6.    On the Proxy Details tab, be sure that Manual Proxy Configuration is selected and then complete the following fields:
For Host or IP Address, enter localhost.
For Port, enter 8157.
Select SOCKS proxy
Select SOCKS v5.

7.    Choose the URL Patterns tab.

8.    Choose Add new pattern and then complete the following fields:
For Pattern Name, enter a name that makes sense to you, such as "VPC Endpoint."
For URL pattern, enter the VPC endpoint for Kibana. Be sure that Whitelist URLs is selected. Be sure that Wildcards is selected.

9.     Choose Save.

Create the SSH tunnel

1.    Run the following command from the local machine that you'll use to access the Kibana dashboard. Replace the following:
mykeypair.pem: the name of the .pem file for the key pair that you specified when you launched the tunnel_ec2 EC2 instance.
public_dns_name: the public DNS of your tunnel_ec2 EC2 instance. For more information, see Viewing DNS Hostnames for Your EC2 Instance.

ssh -i "mykeypair.pem"  ec2-user@public_dns_name -ND 8157

2.    Enter the Kibana endpoint in your browser. The Amazon Cognito login page for Kibana appears.