Use EC2 Instance Connect to provide secure SSH access to EC2 instances with private IP addresses

In this post, I show you how to use Amazon EC2 Instance Connect to use Secure Shell (SSH) to securely access your Amazon Elastic Compute Cloud (Amazon EC2) instances running on private subnets within an Amazon Virtual Private Cloud (Amazon VPC). EC2 Instance Connect provides a simple and secure way to connect to your EC2 instances using one-time SSH keys. It removes the need to share and manage long-term SSH keys. The architecture described is applicable for customers who:

• Require SSH access to EC2 instances running in a private subnet.
• Would like to stop using and managing long-term SSH keys.
• Need to limit source networks that an SSH session can be established from.

EC2 Instance Connect requires access to the public endpoint of the service to perform control plane functions. However, as demonstrated in this post, the SSH traffic from your client to your EC2 instance can remain within your private network. Moreover, EC2 Instance Connect provides rich and granular access control features to enforce the principle of least privilege. It also addresses the difficult task of secure public and private key pair management.

Overview of how EC2 Instance Connect works

EC2 Instance Connect removes the need to share and manage your long-term SSH keys. It uses AWS Identity and Access Management (IAM) and one-time SSH keys to control SSH access to EC2 instances.

An IAM policy attached to an IAM principal controls whether the principal can use EC2 Instance Connect or not. When an authorized IAM principal initiates a connection to an instance using EC2 Instance Connect, the IAM principal sends a one-time SSH public key to the EC2 Instance Connect API. The EC2 Instance Connect Service then sends this SSH public key to the instance metadata service (IMDS) where it remains for 60 seconds. The SSH client must connect to the instance using the private key associated with the public key within that 60 seconds. The standard installation of SSH uses an authorized public key stored on disk. The installation of EC2 Instance Connect on the EC2 instance modifies the SSH daemon to look up the public SSH key using the EC2 instance metadata service. Authentication is successful if the keys match. If no public key is available in the instance metadata service, the SSH daemon also checks the authorized key configured on disk. When using EC2 Instance Connect, it’s recommended that you remove the locally stored authorized key once EC2 Instance Connect is installed on the EC2 instance.

EC2 Instance Connect allows you to connect to EC2 instances using three different methods:

• A browser-based client accessible via the AWS Management Console.
• Your own key and SSH client.
• EC2 Instance Connect command line interface (CLI).

The browser-based client accesses EC2 instances via the AWS Management Console and it works with EC2 instances that have public IPv4 addresses assigned to them. Access to instances with public IP addresses can be locked down not only via IAM policies and source IP conditions but also via Security Groups to the IP ranges used by the EC2 Instance Connect service in a given region, as documented in machine-readable format in the automatically updated in ip-ranges.json file.

On the other hand, EC2 instances without public IP addresses are still accessible via their private IPv4 addresses using either your own SSH client or the EC2 Instance Connect CLI. That private connectivity approach is the focus of this blog post.

For a detailed description of EC2 Instance Connect installation instructions, refer to the EC2 Instance Connect launch blog post.

Example Corp. network topology and IAM policy

Suppose you’ve designed your cloud infrastructure as an extension of your on-premises data center. Access to your EC2 instance is via your corporate network over AWS Direct Connect or AWS Site-to-Site VPN. Furthermore, you need a mechanism to allow authorized users access to EC2 instances at scale, without the need to manage SSH keys.

Figure 1: Example Corp. network topology

You’ve created a standard network topology for using EC2 Instance Connect as depicted in Figure 1. Your on-premises corporate data center connects to the AWS Cloud via Direct Connect. Direct Connect establishes a dedicated network connection between your on-premises network and an AWS Direct Connect partner.

To manage users at scale and across multiple AWS accounts within your organization, you must require that all users authenticate to AWS using AWS Single Sign-On.

There’s a private virtual interface (private VIF) to connect Direct Connect to an Amazon VPC containing two EC2 instances. The Amazon VPC has no internet gateway and no route to the internet and therefore it’s referred to a private Amazon VPC. The private Amazon VPC has a route to your on-premises network via Direct Connect. Each EC2 instance is configured with a private IP address and is protected by a security group configured to allow SSH traffic from the on-premises network range over port 22 (the default SSH port).

A public VIF allows on-premises access to public AWS services over Direct Connect. The source IP address of all the traffic originating from your corporate network to AWS public endpoints is routed over the public VIF. The source IP address of the traffic going from their corporate data center to AWS is translated using NAT (network address translation) to the public IP address of 198.51.100.4. 198.51.100.4 is used as an example public IP address. The NAT is performed by your router deployed in the Direct Connect location.

A system administrator, Martha Rivera, is authorized to access an EC2 instance with instance ID i-00123EXAMPLE using the default Amazon Linux 2 user ec2-user. Martha cannot access any other EC2 instance.

Before Martha can use EC2 Instance Connect, you must create an IAM role that Martha can assume after logging in via AWS SSO.

The IAM role Martha can assume is allowed to access the EC2 instance with instance ID i-00123EXAMPLE. This requirement is enforced by specifying the ARN of the instance under the resource section of the IAM policy.

The security policy also dictates that the role is restricted to sending the public SSH key to EC2 Instance Connect only from within your corporate network via Direct Connect. To achieve that, you attach an IAM policy to the IAM role that Martha can assume. This policy uses the condition key aws:SourceIp.

The IAM policy applied to the role Martha can assume restricts access to instance ID i-00123EXAMPLE. The following is an example of such a policy. The control plane call to API SendSSHPublicKey must be sent over Direct Connect where the source IP is translated to 198.51.100.4 and the OS user name must match ec2-user. This restrictive IAM policy illustrates the level of access granularity you can achieve with EC2 Instance Connect. The EC2 Instance Connect documentation has an example of an IAM policy that uses resource tags to control access to an instance.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2-instance-connect:SendSSHPublicKey"
            ],
            "Resource": [
                "arn:aws:ec2:$REGION:$ACCOUNTID:instance/i-00123EXAMPLE"
            ],
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "198.51.100.0/24"
                    ]
                },
                "StringEquals": {
                   "ec2:osuser": "ec2-user"
                }
            }
       }
    ]
}

Connect to the EC2 instances using EC2 Instance Connect

Now that the role Martha can assume allows them to use EC2 Instance Connect to connect to an EC2 instance with instance ID i-00123EXAMPLE, let’s walk through what happens when they initiate the connection.

Figure 2: Control and data plane when using EC2 Instance Connect

Martha uses the EC2 Instance Connect CLI. When they run mssh i-00123EXAMPLE on the client machine, the EC2 Instance Connect CLI performs the following three operations:

1. Generates a one-time SSH key locally in the client.
2. Pushes the public key to the EC2 Instance Connect service endpoint, which in turn delivers the public key to the IMDS of the instance. This step is indicated by the lower arrow on the preceding figure. It remains in the IMDS for 60 seconds.
3. Connects from the client to the private IP address of the instance via SSH. This step is indicated by the upper arrow on the preceding figure.

For step 2 to take place, the on-premises client machine must have access to the EC2 Instance Connect service endpoint. Because of the IAM condition key aws:SourceIp configured on the IAM role Martha is using, this operation can only be performed successfully if the traffic is routed over the Example Corp. Direct Connect public VIF.

For step 3 to succeed, any firewall, network access list, security groups, or other device performing packet filtering must allow SSH (TCP port 22) from the private IP address of the client machine to the private address of the EC2 instance. That the SSH traffic flows over Direct Connect private VIF and is subject to network ACLs and security groups allows you to enforce the network topology where the SSH session is initiated from.

Alternatively to using the EC2 Instance Connect CLI, Martha could have connected using their own key and SSH client. From an IAM policy and network traffic flow, using their own key and SSH client works the same way as described above.

Conclusion

You can use EC2 Instance Connect to remove the need to manage SSH keys and improve your security posture by providing centralized access control to EC2 instances. The architecture described in this blog post shows how you can use EC2 Instance Connect in deployments where EC2 instances are running within your private subnets.

You can also use EC2 Instance Connect to access your EC2 instances running in public subnets. By defining a network topology and restricting access to the EC2 Instance Connect service using IAM policies, EC2 Instance Connect allows you to enforce where an SSH session originates from.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, start a new thread on the Amazon EC2 forum or contact AWS Support.

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.