Containers

Introducing CDK for Kubernetes

At AWS, we’ve seen customers rapidly adopt Kubernetes to deploy applications globally, train machine learning models at scale, and standardize how they deliver innovation across data centers and the cloud. Using Kubernetes, customers are building automated tooling to replace manual processes, implementing operational pipelines for every piece of their infrastructure, and empowering development teams with the ability to get granular control over how their applications run.

Traditionally, Kubernetes applications are defined with human-readable, static YAML data files which developers write and maintain. Building new applications requires writing a good amount of boilerplate config, copying code from other projects, and applying manual tweaks and customizations. As applications evolve and teams grow, these YAML files become harder to manage. Sharing best practices or making updates involves manual changes and complex migrations.

YAML is an excellent format for describing the desired state of your cluster, but it is does not have primitives for expressing logic and reusable abstractions. There are multiple tools in the Kubernetes ecosystem (kustomize, jsonnet, jkcfg, kubecfg, kubegen, and Pulumi to name a few) which attempt to address these gaps in various ways.

We realized this was exactly the same problem our customers had faced when defining their applications through CloudFormation templates, a problem solved by the AWS Cloud Development Kit (AWS CDK), and that we could apply the same design concepts from the AWS CDK to help all Kubernetes users.

Introducing CDK for Kubernetes

Today I’d like to tell you about the CDK for Kubernetes, or cdk8s, a new open-source project that lets you define Kubernetes applications and reusable components using familiar programming languages. cdk8s (pronounced “cd kates“) lets you use programming languages like TypeScript or Python to generate standard Kubernetes YAML – which means that you can use it to define applications for any Kubernetes cluster running anywhere, both on-premises and the cloud.

cdk8s lets you import both core Kubernetes API objects and Custom Resources (CRDs) as strongly typed classes called “constructs“. This means that you can leverage all the powerful primitives of object-oriented programming to define Kubernetes applications. One of the most powerful capabilities is the ability to compose your own abstractions.

Using cdk8s you can publish common Kubernetes patterns as code libraries, then reference these libraries in any application. This simplifies defining and maintaining applications for all Kubernetes users and builds on top of the Kubernetes declarative API approach while fundamentally respecting its capabilities and flexibility. It also means that you can author Kubernetes applications using the languages, IDEs, tools, and techniques you are familiar with.

Here’s what you need to know about cdk8s:

Works for any cluster cdk8s is environment agnostic. It runs locally on your machine and generates standard Kubernetes YAML data, so you can use it with any Kubernetes cluster running anywhere, including on-premises and the cloud

Imperative approach to declarative state cdk8s code is written using imperative languages but outputs your desired state as pure Kubernetes YAML. This means you can enjoy the expressiveness and simplicity of imperative programming without compromising on the robustness of the declarative desired state approach.

Use any Kubernetes API version and custom resources cdk8s includes a nifty CLI tool that lets you import any version of the Kubernetes API to your project, and update to take advantage of new API versions when you wish. You can also import custom resource definitions.

Language support cdk8s lets you define applications using TypeScript, JavaScript, and Python. We plan to add support for more languages, including Go.

Open source cdk8s is open source and we welcome community contributions. We built cdk8s for the entire Kubernetes community, not just AWS customers.

cdk8s in Action

Let’s take a look at how to define a simple Kubernetes app with cdk8s.

const labels = { app: 'guestbook', tier: 'frontend' };

new k8s.Service(this, 'service', {
  metadata: { labels },
  spec: {
    type: 'LoadBalancer',
    ports: [ { port: 80 } ],
    selector: labels,
  }
});

new k8s.Deployment(this, 'deployment', {
  spec: {
    selector: { matchLabels: labels },
    replicas: 3,
    template: {
      metadata: { labels },
      spec: {
        containers: [
          {
            name: 'php-redis',
            image: 'gcr.io/google-samples/gb-frontend:v4',
            ports: [{ containerPort: 80 }],
            resources: { requests: { cpu: '100m', memory: '100Mi' } }
          }
        ]
      }
    }
  }
});

This is the definition of the Kubernetes guestbook frontend, synthesizing this produces a familiar YAML output that I can apply to my cluster:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: guestbook
    tier: frontend
  name: guestbook-service-23e79b52
spec:
  ports:
    - port: 80
  selector:
    app: guestbook
    tier: frontend
  type: LoadBalancer
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: guestbook-deployment-8b2b7b76
spec:
  replicas: 3
  selector:
    matchLabels:
      app: guestbook
      tier: frontend
  template:
    metadata:
      labels:
        app: guestbook
        tier: frontend
    spec:
      containers:
        - image: gcr.io/google-samples/gb-frontend:v4
          name: php-redis
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: 100m
              memory: 100Mi

Native programming experience

cdk8s includes typed classes for all the Kubernetes APIs. This means that when I write it, I can get all the help from my beloved IDE, which includes code completion, type-safety (for static languages), inline documentation, refactoring tools, and all that jazz:

“kubectl apply” and GitOps Ready

When a cdk8s app is executed it synthesizes standard Kubernetes YAML. This means the workflow and tools you use to deploy your application to the cluster remain the same.

$ cdk8s synth -o dist && kubectl apply -f dist/*
dist/guestbook.k8s.yaml
service/guestbook-service-23e79b52 created
deployment.apps/guestbook-deployment-8b2b7b76 created

This also means that apps defined with cdk8s can neatly be integrated into any standard GitOps workflow. Using cdk8s with GitOps means you can leverage the same workflow for defining your application (write in code, deploy with CICD) as you do for building it. Max Brenner has published some nice posts about integrating cdk8s with Flux and also with Argo CD.

Diving deeper: Constructs

The building blocks of CDK apps are called “constructs” (remember The Matrix?). Constructs can represent anything: from individual Kubernetes resources such as Pods, to complex ideas such as Microservices, or a fully-fledged applications like MongoDB. They can even represent complete systems of multiple applications and tools which model entire clusters.

cdk8s can automatically generate (“import”) constructs for all the core Kubernetes API objects (such as Deployment, Service, ReplicaSet, Pod), as well as from any Custom Resource Definition. In the future, we plan to add support for importing constructs from Helm charts.

Once constructs are imported, we can use them to define Charts. A Chart represents a single Kubernetes YAML manifest. An App is composed of multiple charts that are part of the same project.

In the following example, I’ve declared a chart type HelloChart with a single Kubernetes pod:

class HelloChart extends Chart {
  constructor(scope: Construct, name: string) {
    super(scope, name);

    new k8s.Pod(this, 'hello', {
      spec: {
        containers: [ { name: 'hello', image: 'world' } ]
      }
    });
  }
}

I then define a single instance of this chart in my app:

const app = new App();
new HelloChart(app, 'hello');

// synthesize all charts (manifests) in the app.
app.synth();

The output directory will include a single file called hello.k8s.yaml:

apiVersion: v1
kind: Pod
metadata:
  name: hello-hello-29dae615
spec:
  containers:
    - image: world
      name: hello

Since charts are just normal classes, I can do things like pass in parameters:

interface HelloChartOpts {
  pods: number;
}

class HelloChart extends Chart {
  constructor(scope: Construct, name: string, opts: HelloChartOpts) {
    super(scope, name);

    for (let i = 0; i < opts.pods; ++i) {
      new k8s.Pod(this, 'hello' + i, {
        spec: {
          containers: [ { name: 'hello', image: 'world' } ]
        }
      });
    }
  }
}

And define any number of instances from my chart:

const app = new App();
new HelloChart(app, 'hello-dev', { pods: 1 });
new HelloChart(app, 'hello-prod', { pod: 10000 });
app.synth();

This app will output two files: hello-dev.k8s.yaml with a single pod and hello-prod.k8s.yaml with 10,000 pods (my own little DoS app). Basically one-line of cdk8s generates a YAML with over 50K lines.

Let’s go even deeper: abstraction layers

The cool thing about constructs is that it’s straightforward to compose them into higher-level abstractions.

For example, it’s very common in k8s to have a deployment with a service in front of it, so I can express this using a new construct type called “ServiceDeployment”:

class WebService extends Construct {
  constructor(scope: Construct, name: string, opts: ServiceDeploymentOpts) {
    super(scope, name);

      new k8s.Deployment(this, ...);
      new k8s.Service(this, ...);
    }
  }
}

And then, instantiate it:

new WebService(this, 'service1', { ... });
new WebService(this, 'service2', { ... });

These compositions are also known as abstraction layers. These layers can be stacked on top of each other: the bottom layer in a cdk8s app will normally consist of imported constructs generated from the k8s API or from CRDs, who are then composed into higher level abstractions. Higher layers can abstract the complexity of the API (by providing smart defaults for example) or they can represent opinionated ideas that abstract away major details in favor of a simpler mental model.

Now that I authored my abstractions, I can share them with anyone through a package manager such as npm, PyPI, Maven Central, NuGet or any internal package manager, just like any other class library. This makes it easy to use cdk8s to share best practices across your company or with the community.

Getting creative with rich, object-oriented APIs

Constructs are expressed in my code as object-oriented classes. This means that I can use all the power of object-oriented design to create beautiful and rich APIs for my constructs.

Here is a hypothetical API for the Ambassador API Gateway:

const books = new k8s.Service(this, 'book-collection', ...);
const book = new k8s.Service(this, 'book', ...);

const oauth = new ambassador.Oauth2Filter(this, 'auth', {
  authorizationUrl: 'url',
  // ...
});

const api = new ambassador.Api(this, 'gateway');
api.get('/books', books);
api.get('/books/.*/', book, { prefixRegex: true });
api.post('/books', books, { filter: oauth });
api.put('/books/.*/', book, { filter: oauth, prefixRegex: true });

In this mock up, the ambassador.Api construct exposes a bunch of methods that allow users to describe their route mappings and configuration through a friendly, strongly-typed syntax.

I hope you can see how awesome this can get.

A high-level construct library for Kubernetes

This is day one for cdk8s. We are exploring the implications of designing a rich high-level construct library that will cover the core Kubernetes APIs. The idea is to expose the full feature set of Kubernetes through an awesome class library which is maintained as part of the cdk8s project.

For a taste of what that might look like, consider the common pattern in Kubernetes where you can use a ConfigMap to store the contents of some configuration and make it available to a pod via a volume.

This section in the Kubernetes documentation describes what it entails to do that:

  1. Use kubectl to define a ConfigMap from a directory: 
    kubectl create configmap my-config --from-file=./config
  2. Define the pod through this YAML: 
    apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: registry.k8s.io/busybox
      command: [ "/bin/sh", "-c", "ls /etc/config/" ]
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: my-config
  restartPolicy: Never

This is what this might look like in the high-level library we are designing:

// define a config map with all the files in a local dir
const config = ConfigMap.fromDirectory(this, './config');

// define a pod
const pod = new Pod(this, 'dapi-test-pod');

// add the config map as a volume
const volume = pod.addConfigMapVolume(config);

// add a container to the pod and mount the files
// to /etc/config, and print them on initialization
const container = pod.addContainer('test-container');
container.image = 'registry.k8s.io/busybox';
container.mount(volume, '/etc/config'); // <-- NICE!
container.command = [ "/bin/sh", "-c", "ls /etc/config/" ];
container.restartPolicy = PodRestartPolicy.NEVER;

I hope this gives you a sense of the potential of such high-level API. You can find more details about the high-level design project in the research section of the cdk8s GitHub repo.

cdk8s is now alpha

Today we are announcing that cdk8s is in “alpha” stage. This means that we think it is ready for folks to start to playing with and let us know how we can make this project work even better for their use cases.

It also means that we expect cdk8s to continue to change significantly in the coming months, until we feel it is stable. We will publish a changelog with each release with information about breaking changes and new features.

cdk8s is an open-source project, built for the entire Kubernetes community. It is fully developed on https://github.com/awslabs/cdk8s and we encourage and celebrate all collaboration and contribution.

We’d love to hear what you have to say and welcome contributions on the APIs, developer experience, integration with other tools, documentation, or features.

Here are a few directions we’ve been exploring, along with their GitHub issue links, so you can +1 and participate in the discussion:

We are incredibly excited about this project and truly believe it can make life incredibly productive and fun for developers using Kubernetes.

Get started

To get started, go to cdk8s.io. The best way to get a feel for cdk8s is to pick it up for a little spin through one of the Getting Started guides, either in TypeScript or in Python. These snappy little tutorials will walk you through the first steps of installing cdk8s, using the k8s APIs and even authoring your own custom constructs.

We would also love to connect with you on our Slack channel or Twitter.

Happy coding!

– Elad and Nate