Run Solana nodes on AWS

Solana is a public blockchain built for mass adoption. It’s a high-performance network that is utilized for a range of use cases, including finance, NFTs, payments, and gaming. Solana operates as a single global state machine, and is open and interoperable. In this post, we look into the ways Solana blockchain works, the types of nodes that operate in Solana blockchain network, and dive deeper into the infrastructure needed to run those nodes on Amazon Elastic Compute Cloud (Amazon EC2).

How does the Solana blockchain work?

The Solana blockchain operates with two different types of globally distributed nodes: voting consensus nodes, called validators, and non-voting nodes, called Remote Procedure Call (RPC) nodes. These nodes receive read and write requests, called transactions, to run on-chain programs or retrieve stateful data. Like typical software development workflows, the Solana blockchain operates with similar but different environments called clusters. Each cluster is a distinct grouping of validators and RPC nodes that operate together to process requests and perform consensus operations. There are currently three clusters, each with a different general purpose:

• devnet – This is used as a staging environment for developers to deploy and test programs and core protocol features without the need to spend real Solana SOL tokens. It’s recommended for decentralized client applications (dApps) developers to use devnet when creating applications.
• testnet – This is used by the Solana core contributors to stress test new features on a live network, particularly those related to performance and stability. Testnet tokens are not real and it is not recommended for application developers to use.
• mainnet-beta – This is the production environment for on-chain applications and core protocol features to be accessed by the general public and users of the Solana blockchain. Mainnet tokens are real.

Similar to making HTTP requests to a REST API, dApps will send RPC requests to the non-voting RPC nodes. If the client’s request is only performing a read operation on the blockchain, like requesting stateful data, the RPC node processes the request and sends the response back to the dApp.

When the dApp sends a request to the Solana blockchain that should update its on-chain state, the RPC node registers the request and forwards it to the validators. The consensus validators will then do integrity checks, recency checks, and ensure the correct users have cryptographically signed the request.

Validators will vote on which transactions are valid until consensus is reached. Then the on-chain state changes are performed and the transactions are added into the Solana ledger for permanent storage. The RPC node will then deliver the response back to the client application.

Solana node types on AWS

The configuration of your compute and storage may be different for the types of Solana nodes you run. Let’s review the types of Solana nodes before discussing their infrastructure requirements in the next section.

Consensus

Consensus nodes support proof-of-stake consensus on Solana mainnet-beta and testnet clusters. They’re run by operators to either use their own stake or accept delegated stakes from other holders of Solana SOL tokens. To set up a new consensus node, you first need to generate cryptographic keys that will be used in the process. After the keys are generated with the standard solana-keygen tool, you need to keep a backup of your private secret in a safe place. It is also important to keep in mind that the more SOL tokens are staked through a consensus node, the more outgoing traffic it generates.

Base RPC node without secondary indexes

Base RPC nodes can process the majority of JSON RPC API method calls from your dApps, except those that trigger a scan operation to the entire account set, like getProgramAccounts, and queries specific to tokens created with the Solana Program Library (SPL): getTokenAccountsByDelegate, getTokenAccountBalance, getTokenAccountsByOwner, getTokenLargestAccounts, and getTokenSupply. These node types can use the same infrastructure as the consensus nodes, but instead of validating transactions, they will expose HTTP and WebSocket endpoints for your dApp to interact with the node through the JSON RPC API and RPC Pub/Sub, respectively.

Extended RPC node with secondary indexes

Extended RPC nodes with secondary indexes can process requests for all JSON RPC API methods we’ve mentioned. To process those extra queries faster, the nodes build and maintain additional secondary indexes, which might require from 512 GB to 1 TB of RAM.

Deployment architecture for Solana blockchain nodes on AWS

You can deploy all three node types as standalone single nodes. The single-node setup is helpful for those who would like to stake and validate transactions with the consensus node type, or run an RPC node for non-production needs. To support production-grade dApps, you can use a highly available (HA) setup with multiple RPC nodes managed by an Auto Scaling group and run behind an Application Load Balancer. The HA setup is typically needed only for RPC nodes because consensus nodes don’t serve RPC requests and their availability will not affect the functions of your dApps. You can deploy all three node types and both single and HA setups with the AWS Cloud Development Kit (AWS CDK) application for Solana in the AWS Blockchain Node Runners initiative. Let’s review the key components of both setups.

Single-node deployment architecture

The following diagram illustrates the single-node deployment architecture.

Here is how it works:

1. A Solana node deployed in the default VPC continuously synchronizes with the rest of nodes on Solana clusters through an internet gateway.
2. If you deployed Solana RPC node, it is used by dApps or development tools internally from within the default VPC. The JSON RPC API is not exposed to the internet directly to protect nodes from unauthorized access. In case you deploy Consensus node, this part is not needed because it will not serve JSON RPC API outside of localhost.
3. The Solana node uses all required secrets locally, but stores a copy in AWS Secrets Manager as secure backup.
4. The Solana node sends various monitoring metrics for both EC2 and Solana nodes to Amazon CloudWatch.

Highly available node deployment architecture

The following diagram illustrates the HA node deployment architecture.

The solution works like this:

1. A set of Solana nodes are deployed within the Auto Scaling group in the default VPC and continuously synchronizes with the rest of nodes on Solana clusters through an internet gateway.
2. The Solana nodes are accessed by dApps or development tools internally through an Application Load Balancer. The JSON RPC API is not exposed to the internet to protect nodes from unauthorized access. In production scenarios dApps need to handle user authentication and API protection, like in the following example for dApps on AWS.
3. The Solana nodes use all required secrets locally, but also store a copy in Secrets Manager as secure backup.
4. The Solana nodes send various monitoring metrics for both EC2 and Solana nodes to CloudWatch.

Every 30 seconds, the Application Load Balancer automatically sends health check requests to each of the Solana nodes within the Auto Scaling group. By default, if a node is not responding to three such requests in a row, the Auto Scaling group terminates the EC2 instance running it and provisions another one as a replacement. If required, you can also add more conditions for the Auto Scaling group to elastically adjust the number of nodes based on monitoring parameters aggregated in CloudWatch to address changes in demand.

Recommended infrastructure for Solana node types on AWS

Generally, Solana community usually recommends using AMD EPYC-powered compute instances, but you can also build Solana validator binaries for ARM-based Amazon Elastic Compute Cloud (Amazon EC2) instances powered by AWS Graviton3. Based on experience with running and testing Solana nodes on AWS we have put together the following table. It summarizes different infrastructure configurations for Solana nodes depending on their role. You can also use the experimental options are based on EC2 instance types powered by AWS Graviton 3 processors for development and testing purposes.

For the RPC nodes, the outgoing traffic to maintain the peer-to-peer (p2p) communications is lower than for consensus nodes. We recommend that you connect to the nodes’ RPC endpoints from within AWS Cloud and do not expose them to the internet for better security and cost-efficiency.

Conclusion

In this post, we reviewed how Solana blockchain operates with three public clusters for testing and production use, and with two node types used for validating transactions and processing RPC API requests from dApps. We introduced the three configuration options for Solana nodes on AWS: consensus, base RPC, and extended RPC nodes. Finally, we reviewed cost-effective infrastructure configurations to achieve good performance and dove deeper into two deployment options, which you can try yourself with Solana blueprint for AWS Blockchain Node Runners initiative. If you have further questions, please ask them on AWS re:Post with the tag “blockchain”, Solana on StackExchange , or reach out to the Solana community.

About the Authors

Nikolay Vlasov is a Senior Solutions Architect with the AWS Worldwide Specialist Solutions Architect organization, focused on blockchain-related workloads. He helps customers to run workloads for decentralized web and ledger technologies on AWS.

Nick Frostbutter is a Developer Advocate at the Solana Foundation, where he focuses on improving developer experience, documentation, and educational content in the Solana blockchain ecosystem. As a developer advocate, he creates proof-of-concept applications for the emerging technology and protocols on Solana.