Securing Kubecost access with Amazon Cognito
Kubecost provides real-time cost visibility and insights for teams using Kubernetes. It has an intuitive dashboard to help you understand and analyze the costs of running your workloads in a Kubernetes cluster. Kubecost is built on OpenCost, which was recently accepted as a Cloud Native Computing Foundation (CNCF) Sandbox project, and is actively supported by AWS.
Amazon EKS optimized bundle of Kubecost
Earlier last year, Amazon Elastic Kubernetes Service (Amazon EKS) announced the availability of an Amazon EKS-optimized bundle of Kubecost for cluster cost visibility. The bundle is available to customers free of charge and includes Kubecost troubleshooting support. Kubernetes platform administrators and finance leaders can use Kubecost to visualize a breakdown of their Amazon EKS charges, allocate costs, and chargeback organizational units (e.g., application teams). Kubecost gives internal teams and business units transparent and accurate cost data based on AWS bill. Customers can also get personalized suggestions for cost optimization tailored to their infrastructure environment and usage patterns.
Using Kubecost’s intuitive dashboard, customers can monitor, analyze, and allocate cluster costs. When customers deploy Kubecost in a cluster, the dashboard is secured by NGINX basic authentication, which isn’t recommended in production environments. This post shows how to make the dashboard accessible to external audiences, such as finance leaders, and secure access using Amazon Cognito.
We make the Kubecost dashboard accessible outside the cluster by exposing it using an ingress, which uses Application Load Balancer (ALB). Integrating Amazon Cognito with the ALB, the solution adds support for authenticating and authorizing users to the Kubecost dashboard. To learn more about how ALB and Cognito integrate, please see How to use Application Load Balancer and Amazon Cognito to authenticate users for your Kubernetes web apps.
- Application Load Balancer, Amazon Cognito, and a Transport Layer Security (TLS) Certificate on AWS Certificate Manager (ACM) with Amazon Route 53 hosted zone to authenticate users to Kubecost
- Deployment of Kubecost application using Kubecost add-on for EKS CDK Blueprints
- Kubernetes Ingress with annotations for Amazon Cognito and a TLS Certificate (using Amazon Certificate Manager) for securely authenticating user to Kubecost
Customers can use this pattern to manage multiple clusters across environments with GitOps. Please see Continuous Deployment and GitOps delivery with Amazon EKS Blueprints and ArgoCD to learn about GitOps-driven delivery using EKS Blueprints Patterns.
The secure ingress-auth-cognito Cloud Development Kit (CDK) pattern includes an Amazon EKS cluster configuration, compute capacity configuration, and add-ons required by Kubecost.
You need the following to complete the steps in this post:
- AWS Command Line Interface (AWS CLI) version 2
- AWS CDK version 2.80.0 or later
- Node version 20.0.0 or later
- NPM version 8.19.2 or later
- kubectl version 1.24 or later
- An Amazon Route 53 public hosted zone
Let’s start by setting a few environment variables:
Clone the cdk-eks-blueprints-patterns repository and install dependency packages. This repository contains CDK v2 code written in TypeScript.
The secure ingress-auth-cognito EKS Blueprints pattern is at lib/secure-ingress-auth-cognito/index.ts. In this file, you can find the blueprint definition with all the configurations above using the blueprints.EksBlueprint.builder() method.
The first step to any CDK deployment is bootstrapping the environment. Bootstrapping is the process of provisioning resources for the AWS CDK before you can deploy AWS CDK applications into an AWS environment (an AWS environment is a combination of an AWS account and Region). If you already use CDK in a region, then you don’t need to repeat the bootstrapping process.
Execute the commands below to bootstrap the AWS environment in your Region:
Deploy Kubecost with secured access
In this solution, we’ll allow access to the Kubecost dashboard based on user email addresses. You can control access to the dashboard by allow-listing an entire domain or individual email addresses.
Users are required to sign-up before they can access the Kubecost dashboard. The pre sign-up Lambda trigger only allows sign-ups when the user’s email domain matches the allow-listed domains. When users sign-up, Amazon Cognito sends a verification code to their email address. Users have to verify access (using the one time valid code) to their email before they get access to the dashboard.
First, we’ll create an AWS Systems Manager (SSM) parameter to store the value of the email domain that users use to sign up. Next, we’ll create an environment variable to store the domain name that hosts the Kubecost dashboard. The email domain and the domain used to host the Kubecost dashboard can be same or different. For example, you may choose to host the dashboard at kubecost.myorg.mycompany.com and use firstname.lastname@example.org to login to the dashboard.
Create below parameters with allowed email addresses and domains in the AWS Systems Manager Parameter Store:
If you’d like to limit access to the dashboard by email addresses, then you can also create a parameter to store allowed email addresses and add a logic to the pre authentication Lambda trigger as shown here.
Next, create a secret in AWS Secrets Manager that you’ll use to access ArgoCD. The argo-admin-password secret must be defined as plain text (not key/value):
The CDK code expects the allowed domain and subdomain names in the CDK context file (cdk.json).
Create two environment variables. The PARENT_HOSTED_ZONE variable contains the name of your Route 53 public hosted zone. The DEV_SUBZONE_NAME will be the address for your Kubecost dashboard.
Generate the cdk.json file:
Run the below command from the root of this repository to deploy the solution:
This blueprint will deploy the following:
- Amazon Virtual Private Cloud (Amazon VPC) with public and private subnets, Network Address Translation (NAT) gateways in each availability zone (AZ), and an Internet Gateway
- An Amazon EKS cluster with the following Kubernetes add-ons
- Amazon Cognito user pool, user pool client, domain and also pre-sign-up and pre-authentication lambda triggers to run custom logic to validate users before allowing them to either sign-up or authentication.
Once the deployment is complete, you will see the output similar to shown below in your terminal:
To update your Kubernetes configuration for your new cluster, copy and run the aws eks update-kubeconfig command (the second command in the output) in your terminal.
Validate the access to your Amazon EKS cluster using below kubectl listing all namespaces:
You should see the following namespaces in the cluster:
The stack deploys Kubecost resources in the kubecost namespace.
Testing the authentication
Point your browsers to the URL you associated with the DEV_SUBZONE_NAME key from the CDK context to access the Kubecost dashboard.
The value is also stored as an environment variable:
Your browser will be redirected to an Amazon Cognito hosted User Interface (UI) sign-in page. Since this is your first time accessing the application, select sign up.
The Pre sign-up AWS Lambda trigger for Amazon Cognito User pool is configured to allow users to register only from certain allow-listed email domains. The allow-listed email domains are configured as an environmental variable in the AWS Lambda function. Let us try sign up a new user using an email id, whose domain is not part of the allow list.
You’ll get an error since the domain is not allow-listed.
Let’s sign up as a new user with one of the allow-listed email domains. This time, you’ll get a prompt to confirm your account. Get the verification code sent to your email and confirm your account.
After verifying email address, sign in to access the Kubecost dashboard
Once you sign in, the ALB will redirect you to the Kubecost dashboard:
You continue to incur cost until deleting the infrastructure that you created for this post. Use the commands below to delete resources created during this post:
In this post we showed you how to secure the Kubecost dashboard while making it accessible to users without needing access to the Kubernetes cluster. We used an ALB to expose the dashboard and secured access using Cognito. We also created a record in Route 53 so users can easily access the dashboard.
We used Cognito user pool to store user information. If you already have an identity provider that provides OpenID Connect (OIDC) or SAML 2.0 support, then you can integrate it with Cognito to skip the sign-up and sign-in to the Kubecost dashboard.