Containers

Policy management in Amazon EKS using jsPolicy

Introduction

jsPolicy is an open-source framework for managing validating or mutating admission control policies for Amazon Elastic Kubernetes Service (Amazon EKS) clusters using JavaScript (or TypeScript), which is similar to the way AWS Identity and Access Management (IAM) manages AWS accounts and resource access. It’s also possible to write the entire jsPolicy in a separate file and load the policy from there. jsPolicy offers built-in functions to reduce policy development effort, and testing frameworks like Mocha and Jest can be used to test a policy’s behavior.

This post will walk you through deploying jsPolicy into an Amazon EKS cluster and implementing two policies—one to deny deployments of resources into the default namespace and another to only allow container images from Amazon Elastic Container Registry (Amazon ECR) or the Amazon container image registries.

Prerequisites

For the walk through in this post, the following prerequisites are required:

Solution overview

These are the steps presented in the following sections:

  • Getting started
  • Deploying the Amazon EKS cluster
  • Configuring jsPolicy
  • Creating and testing policies using jsPolicy

Getting started

Before we can get started, we need to set up an Amazon EKS cluster. We use eksctl with the cluster config file mechanism.

Deploy the Amazon EKS cluster

With the necessary tools installed, launch the Amazon EKS cluster. In the following example, the Amazon EKS cluster is deployed to the US East (Ohio) Region (us-east-2). However, the AWS_REGION may be configured for any approved AWS Region where Amazon EKS is enabled.

Export the region and account ID:

export AWS_REGION=us-east-2
export ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)

Once the region is exported, create the ClusterConfig as follows:

cat >cluster.yaml <<EOF
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: jsPolicy
  region: ${AWS_REGION}
nodeGroups:
  - name: ng-1
    desiredCapacity: 2
EOF

After the ClusterConfig is created, create the cluster using the eksctl create cluster command:

eksctl create cluster -f cluster.yaml

The eksctl tool needs approximately 15 minutes to build the Amazon EKS cluster and reach a ready state. In order to use jsPolicy, it must be installed on the Amazon EKS cluster using Helm, which must first be installed on the local machine.

Update kubeconfig

Once the Amazon EKS cluster is built and is in a ready state, update the kubeconfig file to access the cluster:

aws eks update-kubeconfig --region us-east-2 --name jsPolicy

Set up jsPolicy

Install jsPolicy on the Amazon EKS cluster in its own namespace:

helm repo update
helm install jspolicy -n jspolicy --create-namespace --repo https://charts.loft.sh  --generate-name

Validate if the jsPolicy pods are running:

kubectl get pods -n jspolicy

Create and test policies

Create a policy to prevent deployment to the default namespace. The following policy prevents deployments to the default Kubernetes namespace:

cat > deny-default-ns-policy.yaml << EOF
apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-default-namespace.example.com"
spec:
  type: Validating
  operations: ["CREATE"]
  resources: ["*"]
  scope: Namespaced
  javascript: |
    if (request.namespace === "default") {

      deny("Creation of resources within the default namespace is not    allowed!");

     }
EOF

Deploy the policy:

kubectl create -f deny-default-ns-policy.yaml

Test the policy to deny namespace deployments

Create an nginx pod in the default namespace:

Cat > nginx.yaml << EOF
apiVersion: v1
kind: Pod
metadata:
  name: web
  namespace: default
  labels:
    role: web
spec:
  containers:
  - name: web
    image: nginx
    ports:
    - name: http
      containerPort: 80
      protocol: TCP
EOF

Run the nginx pod to test it:

kubectl create -f nginx.yaml

The cluster returns the following error, preventing deployment to the default namespace:

Error from server (Forbidden): error when creating "nginx.yaml": admission webhook "deny-default-namespace.example.com" denied the request: Creation of resources within the default namespace is not allowed!

Create another namespace called web:

kubectl create ns web

Modify the namespace in the nginx.yaml that was created prior to this from default to web and deploy it again:

kubectl create -f nginx.yaml

Create the policy for the allowed list of Amazon ECR repositories

This policy denies pod images not originating from the allow list of Amazon ECR repositories. The first two entries in the policy belong to the account owner and that belonging to the Amazon EKS Amazon ECR repository:

cat > enforce-image-registry.yaml << EOF
apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-untrusted-registries.example.com"
spec:
  type: Validating
  operations: ["CREATE", "UPDATE"]
  resources: ["pods", "deployments", "statefulsets"]
  javascript: |
    const registries = ["602401143452.dkr.ecr.us-east-2.amazonaws.com", "${ACCOUNT_ID}.dkr.ecr.us-east-2.amazonaws.com"]

    // Use template.spec if defined (for Deployments and StatefulSets), or use spec otherwise (for Pods)
    podSpec = request.object?.spec?.template?.spec || request.object?.spec

    podSpec?.containers?.forEach(function(container, index) {
        if (!registries.includes(container.image.split('/')[0])) {
            deny("Field spec.containers[" + index + "].image must be pulled from  " + registries.toString())
        }
    })

    podSpec?.initContainers?.forEach(function(initContainer, index) {
        let imageRegistry =  initContainer.image.split('/')[0] 
        if (!registries.includes(initContainer.image.split('/')[0])) {
            errors.push("Field spec.initContainers[" + index + "].image must match regex: " + registries.toString())
        }
    })
EOF

Deploy the policy:

kubectl create -f enforce-image-registry.yaml

Test the image registry of the allowed list policy

Create an nginx deployment in the web namespace using the nginx image from Docker Hub:

cat > nginx_deployment.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: web
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          ports:
            - containerPort: 80
EOF

Create the nginx deployment to test:

kubectl create -f nginx_deployment.yaml

The cluster returns the following error, indicating that the image should only be pulled from 602401143452.dkr.ecr.us-east-2.amazonaws.com (Amazon EKS Amazon ECR registry) or XXXXXXXX.dkr.ecr.us-east-2.amazonaws.com, where XXXXXXXX is your AWS account number:

Error from server (Forbidden): error when creating "nginx_deployment.yaml": admission webhook "always-pull-latest-image.example.com" denied the request: Field spec.containers[0].image must be pulled from 602401143452.dkr.ecr.us-east-2.amazonaws.com,XXXXXXXX.dkr.ecr.us-east-2.amazonaws.com

Now tag the nginx image with our own registry, push the image, and try deploying from our own registry. For this repository, you need to create it in Amazon ECR:

aws ecr create-repository --repository-name jspolicy

Identify the name of the repository and the tag that the image uses:

export REPOSITORY=$(aws ecr describe-repositories --repository-name jspolicy --query "repositories[0].repositoryUri" --output text)

Pull the nginx image from Docker Hub locally so it can be appropriately retagged:

docker pull nginx

Retag latest with the repository name and push this image to Amazon ECR. Before this can be done, login to Amazon ECR:

aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com

Now push the image with the latest tag to Amazon ECR:

docker tag nginx ${REPOSITORY}/nginx:latest
docker push ${REPOSITORY}/nginx:latest

Now update the nginx deployment manifest file to use the image from Amazon ECR and deploy it:

cat > nginx_deployment.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: web
  labels:
    app: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: ${REPOSITORY}/nginx:latest
          ports:
            - containerPort: 80
EOF

Now create the deployment again:

kubectl create -f nginx_deployment.yaml
deployment.apps/nginx-deployment created

Cleanup

To avoid incurring future charges, delete all the resources that were deployed earlier:

kubectl delete -f nginx.yaml
kubectl delete -f nginx_deployment.yaml
kubectl delete namespace jspolicy
kubectl delete namespace web
aws ecr delete-repository --repository-name jspolicy --force
eksctl delete cluster -f cluster.yaml

Conclusion

If you have previous experience using JavaScript or TypeScript, then you can easily define and deploy custom validating or mutating admission control policies with jsPolicy. We have shown the process in this post. Review the Amazon EKS Best Practices Guide for Security to implement an optimized security strategy for your cluster. To learn more about jsPolicy, check out the jsPolicy documentation and get involved with the jsPolicy community.