AWS Developer Tools Blog

Build and Deploy .Net Core WebAPI Container to Amazon EKS using CDK & cdk8s

In this blog, we will leverage the development capabilities of the CDK for Kubernetes framework also known as cdk8s along with the AWS Cloud Development Kit (AWS CDK) framework to provision infrastructure through AWS CloudFormation.

cdk8s allows us to define Kubernetes apps and components using familiar languages. cdk8s is an open-source software development framework for defining Kubernetes applications and reusable abstractions using familiar programming languages and rich object-oriented APIs. cdk8s apps synthesize into standard Kubernetes manifests which can be applied to any Kubernetes cluster. cdk8s lets you define applications using Typescript, JavaScript, and Python. In this blog we will use Python.

The AWS CDK is an open source software development framework to model and provision your cloud application resources using familiar programming languages, including TypeScript, JavaScript, Python, C# and Java.

For the solution in this blog, we will use C# for the infrastructure code. Completing this walkthrough successfully would take you about couple hours (including installing pre-requisites etc.), so plan accordingly.

Let’s get started!

At a high-level, we will:

  1. Create a simple TODO, Microsoft .NET Core Web API application and integrate with Amazon Aurora Serverless database, AWS SDK Package like SSM into it.
  2. Use AWS CDK to define the Infrastructure resources required for the application.
  3. Use cdk8s to define, deploy and run the application within the created Kubernetes cluster (Created above by CDK)
  4. Use Elastic Kubernetes Service, Elastic Container Registry (ECR), Amazon Systems Manager (SSM) (maintains the Aurora DB Credentials).
  5. Use Amazon Aurora Database (Serverless) as the backend.

Creating the infrastructure described above will result in charges beyond free tier. So, review the pricing section below for service-specific details and make sure to clean up the built infrastructure to avoid any recurring cost.

cdk-cdk8s-architecture

cdk-cdk8s-architecture

The Github source code includes a “cdk8s” folder where the .NET application (docker container WebAPI in ECR) will be deployed and run in the Kubernetes cluster. “cdk” folder contains the AWS Cloud Development Kit (CDK) solution (C# .Net Core) to build the infrastructure. This solution constructs the AWS infrastructure where the “webapi” (.NET Core Web api) is packaged, built as an artifact and pushed to AWS ECR. The .NET project sample uses AWS SDK, Mysql data packages to connect to MySQL and interact with Amazon Aurora database. The exposed Web API endpoint makes HTTP calls (GET & POST) to add/retrieve TODOs. The end user can use any http get/put tool like curl or UI tools like Google Chrome ARC Rest Client or POSTMAN to validate the changes.

Overview of the AWS services used in this solution

  • Amazon Aurora, a MySQL and PostgreSQL-compatible relational database is used as the backend for the purpose of this project.
  • Amazon Elastic Kubernetes Service is a fully managed Kubernetes service. EKS runs upstream Kubernetes and is certified Kubernetes conformant so you can leverage all benefits of open source tooling from the community. You can also easily migrate any standard Kubernetes application to EKS without needing to refactor your code.In this example we use cdk8s to deploy the K8s services and pods The code is provided as part of the solution.
  • Amazon Elastic Container Registry, the AWS provided Docker container registry is used by EKS Managed worker nodes, simplifying the development to production workflow.

Prerequisites

We will use Docker Containers to deploy the Microsoft .NET Web API. The following are required to setup your development environment:

  1. Python >=3.7
  2. AWS CLI
  3. .NET Core:
    1. Web API application was built using Microsoft .NET core 3.1
    2. Please refer Microsoft Documentation for installation.
  4. Docker
    1. Install Docker based on your OS.
    2. Make sure the docker daemon is running
  5. Kubectl
  6. AWS CDK >= 1.58.0
  7. AWS cdk8s
  8. Additionally, we use AWS SDK, MySql data packages for the Microsoft .NET project and have added them as nuget packages to the solution. In this example, we use Mysql data to connect to MySql/Aurora database and also AWS SDK Systems Manager to connect to Amazon Systems Manager.

The Solution

To provision the infrastructure (and services) and deploy the application, we will start by cloning the sample code from the aws-samples repo on GitHub, run installation scripts (included in the sample code) to setup the infrastructure and deploy the webapi to your AWS Account. We will review and test the application, and finally cleanup the resources (basically teardown what you provisioned).

1. Clone the sample code from the GitHub location.

$ git clone https://github.com/aws-samples/aws-cdk-k8s-dotnet-todo

The git source provided above has a “cdk”, “webapi” and a “cdk8s” folder. “webapi” has the necessary .NET Web API solution. We will use the AWS CDK commands to build the infrastructure and deploy the webapi into EKS. cdk8s code provided (using Python language) defines our Kubernetes chart which creates a webservice (k8s Service and Deployment).

Once the code is downloaded, please take a moment to see how CDK provides a simpler implementation for spinning up an infrastructure using C# code. You may use Visual Studio Code or your favorite choice of IDE to open the folder aws-cdk-k8s-dotnet-todo).
Open the file “/aws-cdk-k8s-dotnet-todo/cdk/src/EksCdk/EksCdkStack.cs”. Code below (provided a snippet from the github solution) spins up a VPC for the required Cidr and number of availability zones. Similarly Open the file “/aws-cdk-k8s-dotnet-todo/cdk8/main.py”. Below snippet creates a Kubernetes chart and creates a webservice.

NOTE: Make sure to replace with your AWS account number (where you are trying to deploy/run this application).

main.py is called by “cdk8s.yaml” when cdk8s synth is invoked (by run_cdk8s.sh“). Windows users may have to change the name to ”main.py“ instead of ”.\main.py“ in the cdk8s.yaml


#!/usr/bin/env python
from constructs import Construct
from cdk8s import App, Chart
from imports import k8s
from webservice import WebService
class MyChart(Chart):
    def __init__(self, scope: Construct, ns: str):
        super().__init__(scope, ns)
        # define resources here
        WebService(self, 'todo-app', image='<YOUR_ACCOUNT_NUMBER>.dkr.ecr.us-east-1.amazonaws.com/todo-app:latest', replicas=1)

Open the file “/aws-cdk-k8s-dotnet-todo/cdk/src/EksCdk/EksCdkStack.cs”. Below snippet creates a Kubernetes chart and creates a webservice.


// This sample snippet creates the EKS Cluster
var cluster = new Cluster(this, Constants.CLUSTER_ID, new ClusterProps {
        MastersRole = clusterAdmin,
        Version = KubernetesVersion.V1_16,
        KubectlEnabled = true,
        DefaultCapacity = 0,
        Vpc = vpc                
    });

2. Build the CDK source code and deploy to AWS Account.

Scripts provided

  • run_infra.sh
  • run_cdks.sh
  • cleanup.sh - NOTE. This will clean up the entire infrastructure. This is needed only when we need to cleanup/destroy the infrastructure created by this blog

Provided “run_infra.sh” script/bash file as part of the code base folder, Make sure to replace with your AWS account number (where you are trying to deploy/run this application). This will create the CDK infrastructure and pushes the WebAPI into the ECR. Additionally the script registers the kube update config for the newly created cluster.

If you would like to perform these steps you can do these manual steps as below

Step 1: Steps to build CDK

  • $ cd aws-cdk-k8s-dotnet-todo\cdk
  • $ dotnet build src
  • $ cdk synth
  • $ cdk bootstrap
  • $ cdk deploy --require-approval never

The above CLI will produce output similar to below. Copy and execute this in the command line. This will update your kube config to connect to the EKS control plane.

Below provided below is a sample only:

EksCdkStack.cdkeksConfigCommand415D5239 = aws eks update-kubeconfig –name cdkeksDB67CD5C-34ca1ef8aef7463c80c3517cc12737da –region $REGION —role-arn arn:aws:iam::$ACCOUNT_NUMBER:role/EksCdkStack-AdminRole38563C57-57FLB39DWVJR

Step 2: Steps to Build and push WebAPI into ECR (todo-app ECR repository created as part of above CDK infrastructure)

  • $ cd aws-cdk-k8s-dotnet-todo\cdk
  • $ dotnet build
  • $ aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT_NUMBER.dkr.ecr.$REGION.amazonaws.com
  • $ docker build -t todo-app .
  • $ docker tag todo-app:latest $ACCOUNT_NUMBER.dkr.ecr.$REGION.amazonaws.com/todo-app:latest
  • $ docker push $ACCOUNT_NUMBER.dkr.ecr.$REGION.amazonaws.com/todo-app:latest

Make sure to update your region and account number above

Step 3: Steps to create Kubernetes service and pods using cdk8s

  • $ cd aws-cdk-k8s-dotnet-todo\cdk8s
  • $ pip install pipenv
  • $ cdk8s import
  • $ pipenv install
  • $ pipenv run cdk8s synth
  • $ kubectl apply -f dist/cdk8s.k8s.yaml

After this is run, review the list/cdk8s.k8s.yaml. cdk8s created k8s yaml that is needed for deploying, loading the image from the ECR. A sample is provided below.

In this case, the generated yaml has a Kubernetes service & a deployment.

apiVersion: v1
kind: Service
metadata:
 name: cdk8s-todo-app-service-4b26805b
....
---
apiVersion: apps/v1
kind: Deployment
metadata:
...
 spec:
 containers:
 - image: REDACTED.dkr.ecr.us-east-1.amazonaws.com/todo-app:latest
 name: app
 ports:
 - containerPort: 8080

Once the Kubernetes objects are created, you can see the created pods and services like below. NOTE This could take sometime to start the ELB cluster with the deployment

  • $ kubectl get pods
  • $ kubectl get services

3. Review and Test: Stack Verification

The .NET code provided(cdk/src/EksCdk/Program.cs) creates the EksCdkStack as coded. Based on the name provided, a CloudFormation stack is built. You will be able to see this new stack in AWS Console > CloudFormation.

Stack creation creates close to 44 resources within a new VPC. Some of them are provided here below for your reference.


 AWS::EC2::EIP | eks-vpc/PublicSubnet2/EIP AWS::EC2::VPC | eks-vpc AWS::EC2::InternetGateway | eks-vpc/IGW AWS::EC2::VPCGatewayAttachment | eks-vpc/VPCGW AWS::EC2::Subnet | eks-vpc/PrivateSubnet1/Subnet AWS::EC2::Subnet | eks-vpc/PublicSubnet2/Subnet AWS::EC2::Subnet | eks-vpc/PublicSubnet1/Subnet AWS::EC2::SecurityGroup | cdk-eks/ControlPlaneSecurityGroup AWS::RDS::DBCluster | Database AWS::EC2::SecurityGroup | db-sg AWS::RDS::DBInstance | Database/Instance1 AWS::RDS::DBInstance | Database/Instance2

At the end of this step, you will create the Amazon Aurora DB table and the EKS Cluster exposed with a Classic LoadBalancer where the .NET Core Web API is deployed & exposed to the outside world. The output of the stack returns the following:

cdk-cdk8s-testing

Sample screenshot shows an api/todo (PUT) operation. Similarly you will be able to do GET operation to retrieve todos.

Once the above CloudFormation stack is created successfully, take a moment to identify the major components. Here is the infrastructure you’d have created —

  • Infrastructure containing VPC, Public & Private Subnet, Route Tables, Internet Gateway, NAT Gateway, Public Load Balancer, EKS Cluster.
  • Other AWS Services – ECR, Amazon Aurora Database Serverless, Systems Manager, CloudWatch Logs.

Using CDK constructs, we have built the above infrastructure and integrated the solution with a Public Load Balancer. The output of this stack will give the API URLs for health check and API validation. As you notice by defining the solution using CDK, you were able to:

  • Use object-oriented techniques to create a model of your system
  • Organize your project into logical modules
  • Code completion within your IDE

Using cdk8s chart, were able to generate the needed Kubernetes deployment and service yaml. The generated yaml is applied to the EKS Cluster and exposed using the classic load balancer.

Let’s test the TODO API using any REST API tools, like Postman, Chrome extension ARC or RestMan.

Set Headers as “Content-type” & “application/json”
Sample request:
{
"Task": "Deploying WebAPI in K8s",
"Status": "WIP"
}

Troubleshooting

  • Issues with running the installation/shell script
    • Windows users – Shell scripts by default opens in a new window and closes once done. To see the execution you can paste the script contents in a windows CMD and shall execute sequentially
    • If you are deploying through the provided installation/cleanup scripts, make sure to have “chmod +x .sh” or “chmod +777 .sh” (Elevate the execution permission of the scripts)
    • Linux Users – Permission issues could arise if you are not running as root user. you may have to “sudo su“ .
  • Error retrieving pods/services/kubectl
  • Check if local Docker is running.
  • If you get unAuthorized error – kubectl get pods error: You must be logged in to the server (Unauthorized) 
    https://aws.amazon.com/premiumsupport/knowledge-center/eks-api-server-unauthorized-error/
  • Redeployment. If you are trying to remove and reinstall manually,
    • Make sure to delete CDK staging directory if you are trying to delete and reinstall the stack. Your directory could be like below cdktoolkit-stagingbucket-guid
    • Make sure to delete the SSM parameter “/Database/Config/AuroraConnectionString”
  • Where can i see the load balancer
    • $ kubectl get svc. – This command can provide the LB url
    • Optionally in AWS Console > EC2 > LoadBalancer
  • My application is not loading or running when I hit the LB url
    • Check if the instances are still registering or in healthy state in AWS Console > EC2 > Load Balancer > Select your load balancer > Instances > Check status.
      • Some of the status you might see are “In Service”, “Out Of Service”, “Registration in progress” etc.,
    • This example uses Aurora Serverless Database. The DB may take time to initialize for the first time
      • Check if AWS Console > RDS > “eks-cdk-aurora-database” is running
      • Select “Query Editor” select the database, enter the credentials. This is provided in SSM “Database/Config/AuroraConnectionString”
      • You may provide a SQL query like below to validate – “select * from ToDos”
    • Check if the pods are running
      • kubectl get pods
      • kubectl describe pod
      • kubectl exec -t -i bash
  • My CDK8s deployment failed
    • Make sure you have the prerequisites versions. ex: pipenv, python, kubectl
    • Check your kubectl pods, services (“ex: kubectl get pods, kubectl get src”) to make sure you are able to connect and view the deployment
    • kube config issues – Open AWS Console> CloudFormation> EksCdkStack > Output. Select the aws update kubeconfig command from the console and run that in your command line (note this is done automatically by the installation run_infra script)
    • Windows users check for your reference of “main.py” in cdk8s.yaml (within “cdk8s” folder). This should be “main.py” instead of “./main.py” 
      language: python app: pipenv run ./main.py imports: - k8s

Pricing

4. Code Cleanup

Run the “cleanup.sh” to delete the created infrastructure

If you would like to do this manually, make sure the following resources are deleted before performing the delete/destroy:

  • Stop the Kubernetes services, deployment, pods
  • Contents of the S3 files are deleted.
  • In AWS Console, look for “CDKToolkit” stack
  • Go to “Resources” tab, select the s3 bucket
  • Select all the contents & delete the contents manually

cleanup can be done using the below CLI commands as well:

  • $ cd aws-cdk-k8s-dotnet-todo\cdk8s
  • $ kubectl delete pods --all
  • $ kubectl delete services --all
  • $ aws ecr delete-repository --repository-name todo-app --force
  • $ cdk destroy --force
  • $ aws cloudformation delete-stack --stack-name CDKToolkit

Conclusion

As you can see, we were able to deploy an ASP.NET Core Web API application that uses various AWS Services. In this post we went through the steps and approach for deploying Microsoft .NET Core application code as containers with infrastructure as code using CDK and deploy the Kubernetes services, pods using cdk8s. cdk8s+ is a library built on top of cdk8s. It is a rich, intent-based class library for using the core Kubernetes API. It includes hand crafted constructs that map to native Kubernetes objects, and expose a richer API with reduced complexity. You can check out more cdk8s examples, patterns, AWS EKS Architecture, and intent-driven APIs using cdk8s+ for Kubernetes objects.

We encourage you to try this example and see for yourself how this overall application design works within AWS. Then, it will just be a matter of replacing your current applications (Web API, MVC, or other Microsoft .NET core application), package them as Docker containers and let the Amazon EKS manage the application efficiently.

If you have any questions/feedback about this blog please provide your comments below!

References

About the Authors
Siva RamaniSivasubramanian Ramani (Siva Ramani) is a Sr Cloud Application Architect at AWS. His expertise is in application optimization, serverless solutions and using Microsoft application workloads with AWS.

 

 

Naveen Balaraman is a Cloud Application Architect at Amazon Web Services. He is passionate about Containers, serverless Applications, Architecting Microservices and helping customers leverage the power of AWS cloud.