AWS Storage Blog

Lift and shift migration of SFTP servers to AWS

At re:Invent 2018, we launched a fully managed service called AWS Transfer for SFTP (AWS SFTP) to enable the migration of file transfer workloads to AWS. We spoke with dozens of customers who were eager to replace their existing SFTP server with a fully managed service backed by storage in the cloud.

One of the challenges was to make this migration without any disruption to their end users (SFTP client users). To meet this objective, we designed features to help “lift and shift” without requiring any changes to their applications or workflows.

Bring your own server configuration

We launched with the ability to integrate with your existing identity provider (IdP) (Internally, we affectionately refer to this as “Bring your own auth”.) We talked about that feature in the Enable password authentication for AWS Transfer for SFTP using AWS Secrets Manager post. In this post, we demonstrate the second part of making this migration seamless, importing your server’s configuration.

An SFTP client uses a server’s identity to decide whether it can be trusted. That is, the client decides whether the server is what it claims to be, and whether the connection is secure. A server’s identity consists of three elements:

  • Domain Name System (DNS) record (for example, example.com)
  • IP address (1.23.45)
  • SSH host key

When migrating your server to AWS SFTP, you can bring along as much of your existing server’s identity as required.

DNS record portability

The most straightforward migration scenario is when you simply create an AWS SFTP server along with a DNS CNAME alias to point at your new server’s endpoint. You can use the AWS SFTP Console to create the CNAME record using a DNS service such as Amazon Route 53, or you can use an alternate DNS provider:

Shows end point configuration for Transfer for SFTP

Before publishing the new DNS record, be sure to import your users’ identities into the new server. Your method depends on whether you’re using the service-managed or custom identity provider type.

In either case, after the credentials have been imported from your on-premises server, you must set up a folder structure in your S3 buckets to mirror the existing home directories. Then, when users connect with their SFTP clients (using the same hostname and credentials that they’re used to), they see the same familiar environment.

IP address portability

In the case where an SFTP user is behind a firewall with strict whitelisting, DNS portability may not be sufficient. The actual IP address associated with the server must be preserved. This is possible in AWS SFTP thanks to the Bring Your Own IP Address (BYOIP) feature of EC2 networking. This lets you bring a public IPv4 address from your on-premises network to your AWS account. You continue to own the address, but AWS advertises it on the internet.

There are several steps to binding your existing IP address to your new AWS SFTP server:

  1. Import the IP addresses into your VPC. For high availability, a best practice is to use multiple IP addresses, one in each Availability Zone. This may not be appropriate, given the constraints of your existing user base.
  2. Create a VPC (AWS PrivateLink) endpoint to be used as your AWS SFTP server endpoint.
  3. Create your AWS SFTP server with the VPC endpoint. If you’ve already created the server, you can change the endpoint type, by stopping the server first.
  4. To bridge the internet and your server’s VPC endpoint, create a Network Load Balancer, with the following values:
    • For Scheme, enter a value of “internet-facing”
    • Add a listener on TCP port 22 (or whichever port your old server was using).
    • In the subnets, assign the IP addresses from step 1.
    • Configure routing, specifying a Target type of “IP” and TCP port
    • Register the targets for the load as the VPC endpoint’s IP addresses, which you can find in the Subnets tab for that endpoint in the VPC Console.

After the load balancer is created, you can connect to it through one of the Elastic IP addresses. Within the VPC, you can also continue to connect directly to the VPC endpoint.

Depending on your use case, you may want to create a DNS record for those Elastic IP addresses without one.

Host key portability

This brings me to the third and final piece of your old server’s “identity,” namely its SSH host key.

SFTP clients typically remember a server’s public key when they first connect to it. If the key ever changes, it may indicate an insecure connection, and the client issues a terrifying warning to that effect.

$ sftp wayne@my-sftp-server.mydomain.com
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
SHA256:uRrhorK4Le+WPBS1ihytaHDHerBB+ZQi2Uq4kkuOrlU.
Please contact your system administrator.
Add correct host key in ~/.ssh/known_hosts to get rid of this message.
Offending RSA key in ~/.ssh/known_hosts:145
Password authentication is disabled to avoid man-in-the-middle attacks.
Keyboard-interactive authentication is disabled to avoid man-in-the-middle attacks.
Permission denied (publickey,password,keyboard-interactive).
Couldn't read packet: Connection reset by peer
$

Fortunately, AWS SFTP recently changed to allow customers to create or update servers with an RSA host key of their choice, instead of using the randomly generated key. Do this with the service API, SDKs, or AWS CLI (not supported in the console).

$ aws transfer update-server --server-id s-a07d4717a7e549629 \
        --host-key file://your-existing-server-rsa-host-key
{
    "ServerId": "s-a07d4717a7e549629"
}

The location of your existing host key depends on which server that you’re using, and the operating system that it’s running on. For example, for OpenSSH servers, find the key in /etc/ssh/ssh_host_rsa_key. This default can be overridden using the “HostKey” directive /etc/ssh/sshd_config. So, if you’re having trouble finding the host key, be sure to look there to see if it’s in a non-standard location.

Carefully protect the private portion of any key pair. This is especially true of host keys. Should a host key ever be compromised, it must be rotated, which would affect all users. For this reason, be sure to delete any temporary copies of the key file as soon as it is uploaded and verified:

$ aws transfer describe-server --server-id s-a07d4717a7e549629 \
        --query Server.HostKeyFingerprint
"SHA256:FU8AqSf/Cu/wxT810IZq5DA74uYV4CGBKuGpsSiz4NY="
$ ssh-keygen -l -f your-existing-server-rsa-host-key
2048 SHA256:FU8AqSf/Cu/wxT810IZq5DA74uYV4CGBKuGpsSiz4NY no comment (RSA)rm old-server-host-key
$ rm your-existing-server-rsa-host-key

That said, we do recommend that you keep a copy of the private key in a safe place. After it’s uploaded, the private key cannot be retrieved from AWS SFTP.

With the old private key installed, the scary message has disappeared:

$ sftp wayne@my-sftp-server.mydomain.com
Connected to my-sftp-server.mydomain.com.
sftp>

Summary

Now your lift and shift migration is complete! Your SFTP users can continue to connect just as they always have, unaware that you have successfully moved your SFTP server to AWS.

Wayne Mesard

Wayne Mesard

Wayne is a lead architect for AWS Storage Gateway and AWS Transfer for SFTP. He codes in Java, thinks in MIPS assembly, and dreams in Lisp.