Using Calico on Amazon EKS Windows Containers

This post was contributed by Anuj Singh, Software Development Engineer and Steven David, Enterprise Solutions Architect.

In this blog post, we are going to walk through a step-by-step process on how to install and use Calico for Windows containers running on Amazon Elastic Kubernetes Service (EKS).

Tigera Calico for Windows is a networking and network security solution for Kubernetes-based Windows workloads. You can move Windows workloads like .NET applications into an EKS environment and Calico can help you manage network policy enforcement. Kubernetes network policies allow you to define rules for how groups of pods are allowed to communicate with each other and other network endpoints.

Calico has been validated in an Amazon EKS Linux environment, and the process is documented here. Today, we deep-dive into support for Calico on EKS Windows worker nodes. Calico on Windows nodes runs as a Windows service in contrast to daemonset on Linux nodes.

Tigera Calico for Windows is generally available with enterprise-grade support via Tigera’s Essentials subscription service. The setup below requires you to obtain a EKS_Optimized Tigera Calico binary file and a license key. Please contact Tigera support for more details.

The steps below will help you enable Calico for Windows by leveraging the Amazon VPC CNI plugin, thus eliminating need for installing any custom CNI plugins.

Prerequisites:

• Install and configure the AWS CLI.
• Set up an EKS cluster with both Linux and Windows worker nodes. Please refer to EKS documentation.
• Configure access for RDP client to your Windows nodes.
• Obtain an EKS_Optimized-TigeraCalicov3.9.5.zip from Tigera.
• Obtain a license key from Tigera.

In this blog we will do the following:

• Install Calico resources on the Kubernetes control plane and Linux worker nodes.
• Apply a license key to use Calico for Windows.
• Setup Calico as Windows services on the EKS Windows worker node(s).
• Run a demo to enforce network policy.

Step 1: Install Calico resources on the EKS control plane and Linux worker node

Using Calico for Windows requires resources like the Calico daemonset, some custom resource definitions and appropriate cluster-role/role-bindings. Using kubectl, create the resources like below.

kubectl apply -f https://eks-windows-calico.s3-us-west-2.amazonaws.com/latest/calico.yaml

Verify that the Calico daemon-set is running.

kubectl get daemonset calico-node --namespace kube-system

Note: This also enables Calico support for any Linux workloads like Stars Policy Demo.

Step 2: Apply a license key to use Calico for Windows

After installing the Calico resources, go ahead and apply the license key to your EKS cluster like below.

kubectl apply -f license-key.yaml

Verify that the license key is applied:

kubectl get licensekeys

Step 3: Setup Calico as Windows services on the EKS Windows worker node(s)

Note: Please make sure that the following changes are applied to all Windows worker nodes on which you would like to install Calico.

3.1 Download and unpack the ‘EKS_Optimized-TigeraCalicov3.9.zip’ on your EKS Windows worker node

Expand-Archive .\EKS_Optimized-TigeraCalicov3.9.5.zip C:\

3.2 Modify config.ps1 for your Windows worker node.

Calico requires a few environment variables to be configured as part of the setup. You will find these configuration in '.\TigeraCalico\config.ps1' . Open this file in a text-editor of your choice.

On line 54, replace value for $env:NODENAME with the private DNS name of your Windows worker node (EC2 instance). Please make sure that you enclose it in "". Keep the remaining environment variables as is and save the config.ps1 file.

3.3 Run the install-calico.ps1 script to install Calico services.

./install-calico.ps1

Note that you may loose connection to the remote desktop session temporarily due to creation of container vSwitch. Once installation is completed, it is recommended to restart kubelet service.

Restart-Service kubelet

3.4 Verify that all Calico services and EKS services are running.

Get-Service | where Name -Like 'Tigera*' 
Get-Service | where Name -Like 'kube*'

You can also verify that the Calico services were installed properly by inspecting logs located at .\TigeraCalico\logs\tiger-node.err and .\TigeraCalico\logs\tigera-felix.err. Ideally, these should be empty with no errors.

Step 4: Run a demo to enforce network policy

4.1 Launch Windows and Linux deployments

Use the pod specification like below to launch Windows and Linux pods on the respective worker nodes.

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: windows-server-iis
spec:
  selector:
    matchLabels:
      app: windows-server-iis
  replicas: 2
  template:
    metadata:
      labels:
        app: windows-server-iis
    spec:
      containers:
      - name: windows-server-iis
        image: mcr.microsoft.com/windows/servercore:1809
        imagePullPolicy: IfNotPresent
        command:
        - powershell.exe
        - -command
        - "Add-WindowsFeature Web-Server; Invoke-WebRequest -UseBasicParsing -Uri 'https://dotnetbinaries.blob.core.windows.net/servicemonitor/2.0.1.6/ServiceMonitor.exe' -OutFile 'C:\\ServiceMonitor.exe'; echo '<html><body><br/><br/><marquee><H1>Hello EKS!!!<H1><marquee></body><html>' > C:\\inetpub\\wwwroot\\default.html; C:\\ServiceMonitor.exe 'w3svc'; "
      nodeSelector:
        kubernetes.io/os: windows
    
--- 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        securityContext:
            privileged: true
      nodeSelector:
        beta.kubernetes.io/os: linux

You can name the above spec file as sample-deployments.yaml and apply like:

kubectl apply -f sample-deployments.yaml

Verify that the pods are in ‘Running’ state.

kubectl get pods -o wide --watch

4.2 Test ping connectivity between pods

We would start by verifying that there is network connectivity among all pods. Inside each pod, ping the other two pods’ IP. Update the <nginx-pod-name> below with the pod name that you see in the cluster.

kubectl exec -it <nginx-pod-name> /bin/bash

Similarly, SSH into one of the Windows pods as well. Update the <windows-pod-name> below with the pod names that you see in the cluster.

kubectl exec -it <windows-pod-name> powershell

4.3 Enforce a network policy to restrict ping connectivity

Apply the network policy specification to deny ping traffic to all pods.

---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: block-icmp
spec:
  order: 200
  selector: all()
  types:
  - Ingress
  - Egress
  ingress:
  - action: Deny
    protocol: ICMP
  - action: Deny
    protocol: ICMPv6
  egress:
  - action: Deny
    protocol: ICMP
  - action: Deny
    protocol: ICMPv6

You can name the above spec file as deny-icmp.yaml and apply like:

kubectl apply -f deny_icmp.yaml

4.4 Test ping connectivity between pods.

Follow step 4.2 and check ping connectivity once more. This time you will see that traffic is denied among the pods.

4.5 Remove the network policy enforced.

Now that you’ve tested the network policy, delete it to test whether ping connectivity is established again.

kubectl delete -f deny-icmp.yaml

Conclusion

That is it! You have just configured your Windows worker nodes with Tigera Calico for Windows.

In this blog post, I showed you how to install and use Tigera Calico on Amazon EKS Windows worker nodes. If you have comments or questions about this blog post, submit them in the comments section below.