OpenGamma improves security and reduces overhead using AWS Transfer Family

OpenGamma provides hosted analytics services to derivative-trading firms in the financial services industry. These firms often face the challenge of adapting to new regulatory standards. Since 2008, new regulation has aimed to improve global financial stability by increasing the requirements on firms to post margin in order to protect against default and systemic failure.

As is often the case with new and complex regulation, this can cause inefficiencies at the heart of an organization’s revenue-generating operations. Compounded by market volatility, these increased margin requirements have created a drain on capital reserves, meaning now more than ever firms must manage their margin proactively.

OpenGamma serves the largest and most sophisticated banks, hedge funds, asset managers, pension funds, and commodity trading firms. We help them to understand, predict, and optimize margin costs by independently replicating the models used to calculate margin requirements. In order to do this, we must process thousands of files every day, and maintain hundreds of connections to counterparties, exchanges, and clearing houses, to ingest the sensitive portfolio, collateral, and market data required for these calculations.

By working with the next generation of managed data transfer and storage solutions like AWS Transfer Family and Amazon S3, OpenGamma has transformed its approach to secure file transfer. By consolidating over 50 container instances into a single managed service, OpenGamma has been able to reduce both operational overhead and infrastructure costs. This has improved system security by eliminating custom servers and reducing the attack surface to a single AWS service built to the highest standards.

In this post, I show how OpenGamma achieved secure file transfer in a multi-tenanted environment using AWS Transfer Family’s AWS Transfer for SFTP service, AWS Key Management Service (AWS KMS), Amazon API Gateway, AWS Lambda, and AWS Secrets Manager. I explain how you can do this with full segregation between tenants, using least privilege policies and defense in depth controls. Furthermore, I discuss how to seamlessly migrate from a previous SFTP server, securely manage credentials, and create per-tenant roles while ensuring we encrypt all files at rest using a dedicated encryption key per tenant.

Solution background and overview

As financial services companies have universally adopted the SFTP protocol for file transfers, one of OpenGamma’s key integration requirements is to support file transfers using this protocol. Previously, OpenGamma hosted Dedicated Instances of an SFTP server running on Amazon ECS for each client choosing to integrate using SFTP. This was the simplest way to guarantee the segregation of data and apply different encryption policies per client. Although we had automated provisioning and thought carefully about security, each new server came with additional infrastructure costs, the overhead of monitoring and patching it, and increased the surface area exposed to the internet. High availability was complicated to achieve, and the list of public IP addresses we had to provide for regular penetration testing was getting longer.

Recently, the industry had lacked a viable serverless solution for secure file transfer that can seamlessly integrate with existing cloud infrastructure and achieve segregation between tenants. This changed with the launch of AWS Transfer Family, which now powers all of OpenGamma’s file transfer requirements. As a fully managed service, AWS Transfer Family has reduced the complexity of OpenGamma’s infrastructure, cut hosting costs, and added support for more file transfer standards. The service has helped us accomplish this, all while increasing security and reliability.

When creating an AWS Transfer Family server, we considered the following:

• We must support fixed ingress IP addresses (for customers that approve outbound connections by IP address) and custom security groups (for us to restrict inbound connections to a customer’s IP address), so we must host our server within a VPC.
• We had a previous server SSH host key we needed to bring, to ensure that we could migrate customers to AWS Transfer Family seamlessly without needing them to update their known_hosts file.
• We have customers who use password-based authentication, and migrating to key-based authentication was not an option for them, so we needed a custom identity provider. We manage all other secrets for our services within AWS Secrets Manager, so it would make sense for us to manage passwords here as well.

Organizations that do not require password-based authentication and are happy to manage keys directly within AWS Transfer for SFTP can avoid the need to bring an identity provider. Additionally, if dynamic IP addresses are sufficient and control of network security groups is not needed, then a “publicly accessible” server can be created (as opposed to a “VPC hosted” server).

We first created a VPC to host the server with subnets for each Availability Zone, and an AWS Transfer Family server within the VPC with an Elastic IP address assigned for each subnet. We then created security groups allowing connections from the IP addresses authorized to access our SFTP server. At this stage, you can also specify a custom hostname that can be managed by Amazon Route 53 or your own name servers rather than using the generated CNAME.

Configuring security settings

Illustrating the core features that enable our use of AWS Transfer Family in a multi-tenanted environment: the ability to segregate users and storage, with default encryption in Amazon S3 providing user-specific encryption of data at rest.

Once we configured an AWS Transfer Family server to use our VPC network, we created our custom identity provider with support for AWS Secrets Manager for management of keys and passwords. We also created Amazon S3 buckets, IAM roles, and KMS keys for our tenants.

We opted to create one IAM role and AWS KMS customer master key (CMK) per tenant. We set it up so that the trust policy of the KMS key only allowed usage from the tenant’s IAM role. It’s important to ensure that AWS Transfer Family has permission to assume the tenant’s IAM role.

Given that AWS Transfer Family can only assume a single role per session, this is a robust way to ensure the segregation of tenant data. Even if a misconfiguration mistakenly granted access to another tenant’s bucket, their role would not grant access to the encryption key needed to decrypt the data, providing a layer of defense.

Encryption of uploads using AWS Transfer for SFTP is achieved using the “Default Encryption” option within Amazon S3. For this reason, we created a bucket per tenant with default encryption enabled, and the tenant’s key specified as the encryption key. The tenant’s IAM role then has a policy attached allowing access to their S3 bucket. This provides an additional layer of defense, ensuring that a tenant’s role only has a view into their own data.

Creating the custom identity provider

We extended the authentication scheme in AWS Transfer Family using our own identity provider. Our identity provider is hosted as an internal REST API hosted on API Gateway, calling into AWS Lambda as described in the documentation on working with identity providers. When a user attempts to log into the SFTP server, AWS Transfer Family provides the user name, password (if provided by the SFTP client), the server ID, the user’s IP address, and the protocol used. The identity provider then responds with an IAM role, an S3 bucket, a home directory, an optional scope-down IAM policy, and permitted SSH public keys (if keys were used for secrets instead of a password).

Secrets are stored as a JSON blob within AWS Secrets Manager with a name like sftp/<aws_transfer_server_id>/username and contain account details such as:

{
  "roleArn": "arn:aws:iam::123456789:role/example_tenant",
  "s3Bucket": "example-tenant-bucket",
  "password": "hash",
  "publicKeys": [
    "ssh-rsa AAAA"
  ]
}

The identity provider Lambda function acts as a thin layer upon AWS Secrets Manager. It retrieves the secret that corresponds to the provided user name, verifies the password (if applicable), and then passes through the role ARN, S3 bucket name, or public keys.

You can use scope-down policies to achieve least privilege policies when tenants do not need the full set of permissions granted by the tenant role. Additional properties in the JSON blob can be included to generate a scope-down policy you can apply when assuming a role for the session.

Testing the identity provider

One of the benefits of API Gateway is that it is easy to test APIs, including the identity provider we just created. To test the API, locate the GET method for the /servers/{serverId}/users/{username}/config endpoint within API Gateway and select Test. Enter the server ID of the AWS Transfer for SFTP server, user name, and optionally provide a password within the Headers section as:

Password:myPassword

API Gateway returns the response from the identity provider, which should contain a scope-down policy, role, home directory (S3 bucket), and optionally SSH public keys.

You can also test the end-to-end configuration, including the identity provider, on the AWS Transfer Family console.

Summary

In this blog post, I have shown how OpenGamma has reduced operational overhead, improved security, and reduced infrastructure costs through our use of AWS services to create a multi-tenanted SFTP server. This is only possible because of AWS Transfer Family’s support for segregating user access and data, with per-tenant encryption at rest. I also covered adding support for passwords and scope-down policies using a custom identity provider, enabling seamless migration of users from an existing SFTP host.

The benefits of leveraging the fully managed AWS Transfer Family service cannot be understated. Having configured this just once, we can rely on AWS to provide ongoing stability, security, and reliability, rather than managing a growing stack of infrastructure ourselves. We can provision servers immediately through AWS API calls, and we benefit from feature upgrades as they become available without the need for patches and maintenance.

I hope this blog post provided some helpful guidance on how you can leverage AWS Transfer Family to create a secure, multi-tenanted SFTP service. If you have any comments or questions, do not hesitate to leave them in the comments section. Thank you for reading.