AWS Partner Network (APN) Blog

Building a Secure, Reliable, and Scalable Chainlink Environment on AWS

By Ludovic François, Founder – TrackIt
By Luc Brulet, Arthur Knoepflin, and Jean-Gaël Choppe, Engineers – TrackIt
By Anshu Kapoor, Partner Management Solutions Architect – AWS

TrackIt-AWS-Partners-2023
TrackIt
Connect with TrackIt-1

A prominent blockchain technology provider sought TrackIt‘s assistance in deploying a scalable Chainlink environment on Amazon Web Services (AWS). TrackIt leveraged a comprehensive suite of AWS services to implement a customized Chainlink workflow for the client.

In this post, we detail the steps required to deploy a secure, reliable, and scalable Chainlink environment on AWS.

TrackIt is an AWS Advanced Tier Services Partner and AWS Marketplace Seller that’s a cloud management, consulting, and software development solutions company that specializes in cloud-native, modern software development; DevOps, infrastructure as code (IaC), serverless, CI/CD, and containerization.

TrackIt also provides an open-source AWS cost management tool that allows users to optimize their costs and resources on AWS.

What is Chainlink?

Chainlink is a decentralized blockchain oracle network built on Ethereum. The network is designed to facilitate the transfer of tamper-proof data from off-chain sources to on-chain smart contracts.

Chainlink leverages multiple nodes to obtain the data required for smart contracts. Nodes provide the data and information from off-blockchain sources to on-blockchain smart contracts via data feeds known as oracles.

Chainlink nodes are connected together using a peer-to-peer network. After acquiring data off-chain with external adapters, one of the nodes is responsible for providing the median calculated data from other nodes to the Ethereum blockchain. This helps to save gas (the cost necessary to perform a transaction on the network) by allowing only one node to save the data to the blockchain.

Once the data is in the blockchain, it can be used by smart contracts to make decisions. As shown in Figure 1, a smart contract can be used as a stop-loss mechanism that automatically recognizes when the price of a coin falls below a specific value. In this case, the contract is automatically executed.

The type of data being processed is the exchange rate of USD/ADA. In this context, it refers to the value of one United States dollar in terms of Cardano cryptocurrency. This data could be fetched from various external sources, such as financial APIs or other trusted data providers.

The workflow using chainlink to import the USD/ADA exchange rate into the blockchain is likely used for decentralized applications (DApps) that require real-time or up-to-date pricing information. By integrating the exchange rate data into the blockchain, DApps can access and utilize this information for various purposes, such as:

  • Financial applications: Smart contracts or DApps that involve payments, lending, borrowing, or trading using Cardano may require the latest USD/ADA exchange rate to calculate transaction values accurately.
  • Stablecoins: Projects that maintain stablecoins pegged to the U.S. dollar may use the USD/ADA rate to ensure the stability of their tokens.

TrackIt-Chainlink-Blockchain-1

Figure 1 – Chainlink workflow gathering ADA/USD rates.

Chainlink Deployment Methods

There are multiple ways to deploy Chainlink on Amazon Web Services (AWS). The AWS Chainlink quick start shows how users can deploy Chainlink using Amazon Elastic Compute Cloud (Amazon EC2).

Chainlink can also be deployed on Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Kubernetes Service (Amazon EKS). This post provides an approach to build a secure, reliable, and scalable Chainlink environment on Amazon EKS.

Chainlink Deployment on Amazon EKS Using Terraform

Deploying Chainlink on Amazon EKS facilitates task scalability. Chainlink requires multiple workloads to run properly (metrics, dashboard, adapters) and the Kubernetes Helm package manager provides an easy way to template and replicate the workload configurations.

In addition, Helm has a robust user community and many configurations already exist in the public forum as Helm charts. Networking is also easier under Kubernetes; once a container networking interface (CNI) is provided, it becomes easy to communicate from one workload to another using domain name system (DNS).

TrackIt has created a custom Helm Chart for Chainlink nodes that deploys the following Kubernetes resources:

  • StatefulSet to deploy the pod.
  • Service to expose Chainlink inside the cluster.
  • Ingress to expose Chainlink outside the cluster.
  • PodDisruptionBudget to ensure proper minimum of available pod.
  • Job to set up pod replication for high availability.
  • Secrets to store the database credential used in a Chainlink environment.

Terraform is an open-source infrastructure as code (IaC) software tool that enables developers to safely and predictably create, change, and improve infrastructure. Terraform enables the creation of multiple environments and helps keep track of infrastructure configurations and changes.

The approach in this article uses multiple Terraform modules supported by the AWS community to deploy the AWS infrastructure. Note that these steps are not intended for a production environment but will help you set up your first chainlink node.

Prerequisites

The basic knowledge needed to run these steps without any slowness are:

  • Ethereum (ETH) wallet creation is automatically created by the chainlink node, but if you have any issue with your node it’s important to save the private key somewhere. Learn more in this tutorial.
  • On the stack configuration, an ETH client is required. There are multiple choices: one can host their own ETH client or subscribe to an ETH client. This post uses https://www.infura.io (the ETH URL is provided by the ETH client).
  • It’s possible to have two different wallets: one for the funds and the second to get the LINK once your node is rewarded by the system.
  • This post uses secrets operations (SOPS) to push encrypted credential files instead of pushing clear credentials onto the repository. Refer to this link for SOPS details and install instructions.
  • This article deploys chainlink adapters for Cryptocompare and Tiingo. Get API keys prior to starting deployment from these links:

Architecture

This architecture contains an Amazon Virtual Private Cloud (VPC) with subnets to host the infrastructure. The EKS cluster with its Kubernetes nodes (using Amazon EC2 instances) are used to host Chainlink and Chainlink adapters. Amazon Aurora with PostgreSQL is used by Chainlink

Other resources have been deployed such as secrets in AWS Secrets Manager to store sensitive information, and an AWS Key Management Service (AWS KMS) key to encrypt secrets with SOPS. Amazon Elastic File Service (Amazon EFS) has also been used as the block storage system which is compatible with EKS and cross-Availability Zone replicated.

An Internal Load Balancer is automatically created by EKS through the Load Balancer Controller, and Amazon Route 53 records are automatically created by ExternalDNS. The virtual private network (VPN) is used to access Chainlink (those parts are not covered in this post).

TrackIt-Chainlink-Blockchain-2

Figure 2 – Solution architecture.

Terraform Setup

The sample code to deploy Chainlink on Amazon EKS is hosted by TrackIt in this GitHub repo. Follow the steps below to deploy Chainlink on EKS in your environment using TrackIt’s sample code.

  • Clone the Git repo to your local system.
cd ~
git clone https://github.com/trackit/aws-eks-chainlink-whitepaper.git
cd aws-eks-chainlink-whitepaper/
  • Create an Amazon Simple Storage Service (Amazon S3) bucket to store Terraform state. Replace bucket_name with your bucket name and aws_region with your AWS region.
aws s3api create-bucket --bucket {{bucket_name}} --region {{aws_region}} --create-bucket-configuration LocationConstraint={{aws_region}}
  • Create an Amazon DynamoDB table to store Terraform state lock. Replace {{table_name}} with your table name and {{region}} with your AWS region.
aws dynamodb create-table --table-name {{table_name}} --attribute-definitions AttributeName=LockID,AttributeType=S --key-schema AttributeName=LockID,KeyType=HASH --billing-mode PAY_PER_REQUEST --region {{region}}
  • Update the provider.tf file in the aws-eks-chainlink-whitepaper/tf directory with the Terraform S3 backend and provider configuration. Refer to the example below, with us-east-1 as the region, terraform-state as the S3 bucket name, and terraform-state-lock as the DynamoDB table name.
terraform {
  required_version = "~> 1.3.9"
  backend "s3" {
    bucket         = "terraform-state"
    key            = "terraform.state"
    region         = "us-east-1"
    dynamodb_table = "terraform-state-lock"
  }

  required_providers {
    sops = {
      source  = "carlpett/sops"
      version = "~> 0.5"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

provider "sops" {}
aws kms create-key
  • Create a new tfvars file for the environment, such as envs/dev.tfvars for the dev environment.
    • Fill in the values for the variables (example.tfvars contains an example for reference).
    • Create the env folder for secrets, such as tf/secrets/dev for dev environment.
mkdir tf/secrets/dev

Secrets: SOPS

  • Create a new file clear-dev-secrets.yaml which will contain the secrets for environment.
  • Fill in the values for the secrets (use clear-secrets.yaml as a reference).
  • Encrypt secrets (use the KMS key ARN for the key created above).
export SOPS_KMS_ARN=arn:aws:kms:REGION:ACCOUNT_ID:key/KMS_KEY_ID
sops -e clear-secrets.yaml > ./tf/secrets/dev/encrypted-secrets.yaml
  • Confirm the encrypted file is the tf/secrets/<env> folder and named is encrypted-secrets.yaml.

Deployment

After completing the setup, follow these steps to deploy the Chainlink node and adapters:

  • Open a terminal and go to the tf folder.
cd ~/aws-eks-chainlink-whitepaper/tf
  • Initialize Terraform.
terraform init
  • (Optional) To use workspace, create it if the workspace does not exist and select it.
terraform workspace new dev && terraform workspace select dev
  • Plan the deployment to verify that the deployment matches configuration.
terraform plan --var-file ../envs/dev.tfvars
  • Deploy the infrastructure.
terraform apply --var-file ../envs/dev.tfvars

Connect to Cluster

  • kubectl is required to connect to the cluster. Install kubectl by following the documentation.
  • Once the infrastructure is deployed, connect to the EKS cluster.
aws eks update-kubeconfig --name <CLUSTER_NAME> --region <AWS_REGION>
kubectl get pods
  • Pods including Chainlink will be visible in the kubectl output. Copy the name of the Chainlink pod, and then use the following command to access the Chainlink user interface (UI) on your machine.
kubectl port-forward <CHAINLINK_POD_NAME> 6688:6688
  • Open your browser and go to http://localhost:6688/ to access the Chainlink UI and fill your user credentials.

Go Further

To customize the Chainlink nodes and adapters, refer to TrackIt’s Chainlink Helm charts documentation.

Cleanup

  • Open a terminal and go to the tf folder.
cd ~/aws-eks-chainlink-whitepaper/tf
  • Destroy Terraform.
terraform destroy --var-file ../envs/dev.tfvars

Known Issues

When destroying the Terraform project, the following error may occur:

│ Error: deleting EC2 EIP (eipalloc-XXXX): disassociating: AuthFailure: You do not have permission to access the specified resource.
│       status code: 400, request id: XXXX

As a workaround, destroy the project again.

Production Considerations

The steps listed here are not intended for production environment. Some additional considerations before deploying in a production environment include implementing additional security measures to encrypt data at rest and in transit, and implementing additional access control.

Conclusion

With more than 70 validated blockchain solutions from partners providing support to all major blockchain protocols—including Sawtooth, DAML, Hyperledger, Ethereum, Corda, and Blockstack—AWS has become the leading vendor for customers looking to deploy blockchain and ledger technology workloads on the cloud.

This post serves as a resource you can use to quickly deploy secure, reliable, and scalable Chainlink environments on AWS.

You can learn more about TrackIt in AWS Marketplace.

.
TrackIt-APN-Blog-Connect-2023
.

TrackIt – AWS Partner Spotlight

TrackIt is an AWS Advanced Tier Services Partner and cloud management, consulting, and software development solutions company that specializes in cloud-native, modern software development; DevOps, IaC, serverless, CI/CD, and containerization.

Contact TrackIt | Partner Overview | AWS Marketplace | Case Studies