Securely generate Ethereum validator keys at low cost using a serverless architecture on AWS

The new proof-of-stake consensus mechanism on Ethereum blockchain network allows network participants to earn rewards without spending a lot of computational power for playing a part in securing the network. Unlike miners, validators don’t need to spend many CPU cycles to produce new blocks. They stake 32 ETH (the native cryptocurrency on Ethereum) to be allowed to attest user transactions and that way generate income through rewards. They also may lose a portion of their funds (slashed) for incorrect behavior, or for their node being unavailable. To avoid penalties, validators need to keep the availability of their infrastructure as high as possible, and it is a good reason why to run it on the cloud. Each validator has a public-private keypair associated with it. The validator keys (the public and private keys) are used to collect, and withdraw the 32 ETH collateral and any rewards that have been collected. The withdrawal feature is available in Ethereum’s Shapella update. Generating and storing validator keys in an unsecure manner poses risks of leaked keys. Other than withdrawal of the collateral by malicious parties, leaked keys can also be used to perform long-range attacks, putting the entire Ethereum network at risk of a history rewrite. Long-range attacks work by creating and maintaining a parallel chain, thus rewriting the entire history of the blockchain. To reconstruct a parallel chain, the keys of past validators are needed. Therefore, in this blog post we will focus on safeguarding the validator keys since their inception.

Solution overview

When generating new Ethereum validator keys, the common practice is to use code written and tested by the community. But in addition to the code itself, the overall solution should:

• Make sure the private key will never be exposed right after generation and always stays encrypted at rest.
• Set the process to get assurance that the code generating the key pair is properly reviewed before deployment, and only the reviewed version can be deployed.
• Be as cost-effective as possible because key generation is a relatively rare process and we don’t need always-on resources to support it.

In the proposed solution, we run staking-deposit-cli tool from the official Ethereum GitHub account in AWS Lambda. staking-deposit-cli tool is used generate EIP-2335 format BLS12-381 keystores (the public and private key-pair) and a corresponding deposit data JSON file. The deposit data JSON file is to be uploaded to Ethereum Staking Launchpad to signify registration of the validator to the Ethereum network. The aforementioned items are generated in-memory in AWS Lambda. We also introduce two personas: release manager and developer, to facilitate the code review process. The release manager is authorized to sign the code shared by developer with a signing profile from AWS Signer. The code signing feature of Lambda ensures that the source code package of the secure key-generation function is signed with the signing profile controlled by the release manager.

The following diagram depicts the key components and how they are used for code deployment and key generation. The accompanying GitHub repository contains the code and instructions for deploying this solution into your own account.

The diagram includes the following high-level steps to deploy the function code, review it, and then use that function to generate keys:

1. Developer packages the source code for the staking-deposit tool and uploads it to the deployment Amazon Simple Storage Service (Amazon S3) bucket.
2. The release manager triggers the code signing process with AWS Signer.
3. AWS Signer creates the signed version of the source code.
4. The release manager triggers the deployment of the signed code package to Lambda.
5. An automated trigger or an individual authorized user triggers the key-generation Lambda function.
6. The Lambda function generates the secrets in-memory and encrypts them using data encryption key within AWS Key Management Service (AWS KMS). In our sample application we pre-generate the KMS data encryption key during the deployment of the solution. The secret generated by the function is JSON-formatted data which consists of the following:
   • An encrypted BLS12-381 keystore
   • A randomly generated password to decrypt the keystore
   • A mnemonic from which the private key is derived from
7. The Lambda function sends the encrypted secrets and the deposit data in base64 format to Amazon DynamoDB. It can be retrieved from the DynamoDB table, decoded, and finally uploaded to the staking launchpad.

Finally, upload the deposit data JSON object to the staking launchpad to finish the staking process. The deposit data JSON object is not encrypted – that’s because it doesn’t contain any sensitive information, so it can be It can be retrieved from the DynamoDB table (see the details in the README of the accompanying GitHub repository)

After our secrets are encrypted and stored safely, we can use it to sign attestations, which we will discuss next.

Key decryption and usage

In our solution, the AWS KMS key policy shouldn’t grant the Lambda function access to perform decryption of the encrypted validator key because the Lambda will never sign attestations. Instead, the key policy should be modified to allow only the protected compute, such as isolated Amazon Elastic Compute Cloud (Amazon EC2) instance or even more isolated AWS Nitro Enclaves, to decrypt the encrypted validator keys in order to sign attestations. The code on those instances performs the following actions:

1. Retrieve the validator private key from DynamoDB as cyphertext.
2. Decrypt the cyphertext with the corresponding data encryption key in AWS KMS.
3. Pass the decrypted validator key to Web3Signer code to sign attestations.

In the following series of blog posts we show the example of how such system will work, so will omit the details here.

The isolated Lambda execution environment for secure key generation

In Lambda, execution environments are isolated from one another and may only be used for one concurrent invocation at a time. Customers and other users can initiate inbound or ingress network communications with an execution environment only through the Lambda’s Invoke API, which triggers the invocation of a Lambda function (services’ data plane). This helps ensure that communications to your execution environment are authenticated and authorized, and that users can interact with its storage or memory only through the code that you deployed. In addition, all customer data stored within the management APIs of Lambda (its control plane) is encrypted at rest through the use of AWS KMS, which is designed to protect it from unauthorized disclosure or tampering. At the same time, Lambda execution environments may be reused across multiple invocations of the same function version for performance reasons. Therefore, the function code in the accompanying sample app regenerates the secrets for every new invocation to make sure they are not reused by subsequent calls.

For audit purposes, you can also enable AWS CloudTrail logs on the AWS account with the deployed Lambda function and use AWS Config to track your function configuration changes.

For more information on Lambda security, refer to Security Overview of AWS Lambda.

Conclusion

In this post, we learned about the new role of validators in the Ethereum network that participate in the proof-of-stake consensus mechanism. We reviewed the requirements for safe key generation on cloud and walked through a sample solution architecture and the process it helps facilitate. We also reviewed the security aspects that Lambda brings into the picture, and how it encompasses security best practices using Lambda.

To learn more about using the encrypted validator keys in a secure environment using AWS Nitro Enclaves read this series of blog posts: part 1, part 2.

About the Authors

Nikolay Vlasov is a Senior Solutions Architect with AWS Worldwide Specialist Solutions Architect organization focused on blockchain-related workloads. He helps clients to turn their ideas into Pilots, Minimally Viable Products, and Production-ready systems based on blockchain technology.

Aldred Halim is a Solutions Architect with AWS Worldwide Specialist Solutions Architect organization. He works closely with customers in designing architectures and building components to ensure success in running blockchain workload on AWS.