[SEO Subhead]
This Guidance demonstrates how to set up a workload accelerator for Amazon Elastic Kubernetes Service (Amazon EKS) using Terraform blueprints. This collection of sample deployments and configurations addresses the challenges often associated with building your first application-ready Amazon EKS cluster. It incorporates a set of pre-configured and integrated tools, add-ons, and best practices to support core capabilities, including automatic scalability, observability, networking, and security. By using this Guidance and the Terraform blueprints, you can accelerate the process of establishing a fully configured, production-ready Amazon EKS cluster to support your workloads, without having to build and maintain the underlying infrastructure.
Note: [Disclaimer]
Architecture Diagram
[Architecture diagram description]
Step 1
Define a per-environment Terraform variable file that controls all environment-specific configurations. This configuration file is used in all steps of the deployment process by all infrastructure as code (IaC) configurations to create different Amazon Elastic Kubernetes Service (Amazon EKS) environments.
Step 2
Apply the environment configuration using Terraform.
Step 3
An Amazon Virtual Private Cloud (Amazon VPC) is provisioned based on the specified configuration. According to best practices for reliability, three Availability Zones (AZs) are configured with corresponding virtual private cloud (VPC) endpoints for access to resources deployed in the private VPC.
Provisioned resources for private clusters, including Amazon Elastic Container Registry (Amazon ECR), Amazon EKS, Amazon Elastic Compute Cloud (Amazon EC2), and Amazon Elastic Block Store (Amazon EBS) are available through the corresponding VPC endpoints.
Step 4
User-facing AWS Identity and Access Management (IAM) roles (cluster admin, admin, editor, and reader) are created for various access levels to Amazon EKS cluster resources, as recommended in Kubernetes security best practices.
Step 5
The Amazon EKS cluster is provisioned with a managed nodes group (MNG) that runs critical cluster add-ons (such as CoreDNS, Karpenter, and AWS Load Balancer Controller through Elastic Load Balancing) on its compute node instances.
Karpenter will manage the compute capacity to other add-ons (as well as business applications that your users deploy) while prioritizing instances powered by AWS Graviton processors for better price performance. Amazon EKS managed elastic network interfaces (ENIs) are deployed in isolated subnets.
Step 6
Other relevant Amazon EKS add-ons (such as cert-manager or External Secrets Operator (ESO)) are deployed based on their configurations defined in the corresponding environment Terraform configuration files (step 1).
Step 7
An AWS managed observability stack is deployed (if configured), including Amazon Managed Service for Prometheus (with an AWS managed collector for Amazon EKS) and Amazon Managed Grafana. In addition, a Grafana operator add-on is deployed with a set of predefined Grafana dashboards to get you started.
Step 8
One or more Amazon EKS clusters with important add-ons (optionally configured with a managed observability stack and IAM role-based access control) are available for your production workload deployment. These clusters’ Kubernetes APIs are exposed through Network Load Balancers.
Step 1
Define a per-environment Terraform variable file that controls all environment-specific configurations. This configuration file is used in all steps of the deployment process by all infrastructure as code (IaC) configurations to create different Amazon Elastic Kubernetes Service (Amazon EKS) environments.
Step 2
Apply the environment configuration using Terraform.
Step 3
An Amazon Virtual Private Cloud (Amazon VPC) is provisioned based on the specified configuration. According to best practices for reliability, three Availability Zones (AZs) are configured with corresponding virtual private cloud (VPC) endpoints for access to resources deployed in the private VPC.
Provisioned resources for private clusters, including Amazon Elastic Container Registry (Amazon ECR), Amazon EKS, Amazon Elastic Compute Cloud (Amazon EC2), and Amazon Elastic Block Store (Amazon EBS) are available through the corresponding VPC endpoints.
Step 4
User-facing AWS Identity and Access Management (IAM) roles (cluster admin, admin, editor, and reader) are created for various access levels to Amazon EKS cluster resources, as recommended in Kubernetes security best practices.
Step 5
The Amazon EKS cluster is provisioned with a managed nodes group (MNG) that runs critical cluster add-ons (such as CoreDNS, Karpenter, and AWS Load Balancer Controller through Elastic Load Balancing) on its compute node instances.
Karpenter will manage the compute capacity to other add-ons (as well as business applications that your users deploy) while prioritizing instances powered by AWS Graviton processors for better price performance. Amazon EKS managed elastic network interfaces (ENIs) are deployed in isolated subnets.
Step 6
Other relevant Amazon EKS add-ons (such as cert-manager or External Secrets Operator (ESO)) are deployed based on their configurations defined in the corresponding environment Terraform configuration files (step 1).
Step 7
An AWS managed observability stack is deployed (if configured), including Amazon Managed Service for Prometheus (with an AWS managed collector for Amazon EKS) and Amazon Managed Grafana. In addition, a Grafana operator add-on is deployed with a set of predefined Grafana dashboards to get you started.
Step 8
One or more Amazon EKS clusters with important add-ons (optionally configured with a managed observability stack and IAM role-based access control) are available for your production workload deployment. These clusters’ Kubernetes APIs are exposed through Network Load Balancers.
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational ExcellenceRead the Operational Excellence whitepaper
This Guidance provisions observability tooling either through an open source software (OSS) or AWS, helping you follow best practices for cross-service configuration. For AWS observability tooling (using Amazon Managed Service for Prometheus and Amazon Managed Grafana), this Guidance creates Grafana dashboards based on the AWS Observability Accelerator project. Additionally, Amazon CloudWatch provides focused log event management. You can use these dashboards and logs to easily monitor utilization trends, quickly identify issues and their root causes, and make data-driven decisions for optimizing operations.
-
SecurityRead the Security whitepaper
This Guidance configures a secured VPC with public, private, and isolated subnets, so when applications are being deployed to Amazon EKS, they cannot be directly accessed externally. You can also request a completely isolated VPC (no internet access). For that use, this Guidance provisions VPC endpoints for the relevant services that are needed for the cluster operations, such as Amazon EC2, Amazon ECR, Amazon EKS, or Amazon EBS. Additionally, you can use IAM to create fine-grained user-facing roles to control access to Amazon EKS clusters.
-
ReliabilityRead the Reliability whitepaper
Amazon EKS scales automatically, and the Kubernetes control plane deploys across multiple AZs to maintain consistent infrastructure health and availability. Additionally, by deploying Amazon Managed Service for Prometheus and Amazon Managed Grafana according to best practices, you can make sure they support high availability and fault tolerance. Finally, the Karpenter automatic scaler for Amazon EKS compute nodes manages the application infrastructure and helps make sure compute node instances are running on the latest Amazon Machine Image (AMI) for the cluster version.
-
Performance EfficiencyRead the Performance Efficiency whitepaper
Karpenter helps right-size instances and automatically scales Amazon EKS cluster compute nodes up and down as needed to match the total resources requested by applications running in the cluster. Additionally, it provisions instances powered by AWS Graviton processors to enhance price performance.
-
Cost OptimizationRead the Cost Optimization whitepaper
The cost for the Amazon EKS cluster is fixed, regardless of size, and is significantly lower than the cost of self-maintaining a secured, highly available, and scalable infrastructure. For compute, Karpenter provisions and allocates resources strictly according to the needs of the application workload so that you don’t have to pay for overprovisioned resources. Additionally, AWS Graviton processors deliver enhanced price performance for the compute nodes.
-
SustainabilityRead the Sustainability whitepaper
Amazon EKS (with Amazon EC2 compute node instances) and Amazon ECR run on AWS, so you do not need to provision your own physical infrastructure. Karpenter automatically scales the cluster’s compute nodes based on demand, and AWS Graviton processors provide up to 40 percent energy efficiency compared to other processors for Amazon EC2 instances, helping you minimize your compute resource footprint and its environmental impact.
Related Content
[Title]
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.