Getting started with Amazon EKS Anywhere on Bare Metal
We are happy to announce the general availability of Amazon EKS Anywhere on Bare Metal. We released Amazon EKS Anywhere in 2021 with support to run on-premises Kubernetes clusters using VMware and today you can use EKS Anywhere to provision clusters without virtualization. The new functionality adds support for managing the full hardware lifecycle to boot, provision, and operate clusters.
Running Kubernetes clusters on top of virtualization technologies can provide flexibility for your compute needs with live migrations, dynamic scaling, and hardware abstractions for the operating system. The new bare metal capability is provided in addition to the existing support for EKS Anywhere running on VMware. Many customers have asked for the ability to run EKS Anywhere directly on hardware with no virtualization. Running Kubernetes on bare metal instances will take full advantage of the servers because the operating system is communicating with the physical hardware. This allows workloads to take advantage of compute accelerators, local storage, and native I/O speeds. The benefits virtualization brings to virtual machines Kubernetes does for workloads without additional abstractions, management, or performance overhead.
EKS Anywhere is fully open source and free to use. It builds on a variety of existing, open-source projects such as, Tinkerbell for provisioning servers, kind for bootstrapping, and Cluster API for Kubernetes lifecycle management. Let’s create our first bare metal cluster.
The minimum specs for an EKS Anywhere server is:
- 4 CPU
- 8 GB memory
- 20 GB available storage
- Network card that can PXE boot
You can run highly available clusters with five servers for the control plane—3 etcd, and 2 Kubernetes API—or you can run a cluster without high availability on a single server with a minimum of 16 GB of memory. Cluster workloads are not scheduled on the control plane, and you can have different node configurations in your cluster as long as they meet the minimum specs.
We have worked with multiple hardware providers and integrators to make sure your on-premises clusters are fully supported.
You can see a list of launch partners near the end of this post.
Create a cluster
To create a bare metal cluster you need to create an inventory file with information about the hardware you want to use. The file is in CSV format and will be used by EKS Anywhere’s bootstrap process to power on and install an operating system to your servers. You can customize the provisioning workflow by adding your own packages, configuration, or files as shown here in the documentation.
The CSV file needs the following information at a minimum:
- Installation disk
- Network configuration
An example CSV file will have these fields:
A full example of the inventory file can be found in the documentation.
Install the EKS Anywhere CLI to generate a cluster configuration. The installation will work on macOS or Linux. Windows and Windows subsystem for Linux (WSL) are not supported. If you would like to manually install the CLI you can read how in the documentation.
Now that we have our hardware configuration and CLI installed we can generate the cluster configuration. This is a declarative specification to configure a cluster.
In the following commands you can replace the
$VARIABLES with your own configuration:
Edit the configuration to add information unique to your cluster and environment. Open the clusterconfig.yaml file in your favorite text editor and edit the metadata.name entry to give the cluster a unique name and modify
controlPlaneConfiguration.endpoint.host entry to an available IP address on your network. This IP address will be the IP address of your Kubernetes API server, but it should not already be in use on your network. It will be a virtual IP address (VIP) that allows for highly available API servers.
You also need to add an IP address for spec.
tinkerbellIP under the
TinkerBellDatacenterConfig object. This IP address will be used for PXE booting machines in the cluster.
You can optionally edit other configurations in the file. Some common configurations include:
- Pod and service CIDR blocks
- sshAuthorizedKeys for control plane and worker nodes
Once the configuration is set, you can begin the cluster creation process. Just like with other providers, EKS Anywhere will run a bootstrap cluster on your local machine to prepare the initial cluster. You can read more about how the bootstrap process works in the documentation.
Use the hardware inventory along with your cluster configuration to create the bare metal cluster.
EKS Anywhere works with out-of-band (OOB) and lights-out management (LOM) hardware in servers. This allows you to control the power of a server over the network even if it is powered off and you don’t have physical access. EKS Anywhere uses the Intelligent Platform Management Interface (IPMI) protocol or Redfish to communicate with your servers and manage their power state to provision them.
If your hardware has an out-of-band management, and you provided the connection information in the inventory file, then EKS Anywhere will automatically boot your hardware, install the operating system, configure Kubernetes, and boot into the full, installed operating system.
You should see output similar to this when the process is done:
You can now use your EKS Anywhere cluster using the generated kubeconfig file in the local directory.
You’ll see the nodes you declared in your inventory file.
You can deploy a test workload to the cluster using:
We have worked with a lot of partners to validate their hardware with EKS Anywhere on Bare Metal. Here is a small list of some of the partners and validated hardware.
LG CNS: Application modernization with containers has become the key competency for the enterprise. Enterprise customers have requirements that address policies for privacy, compliance, and edge use cases. Amazon EKS Anywhere provides the flexibility to accommodate the modernization of diverse customer workloads while extending Amazon Elastic Kubernetes Service (Amazon EKS) capabilities on premises. Furthermore, it is noteworthy that not only VMware but also bare metal options have been added to enable more efficient EKS environments, and consistent management of on-premises EKS environments is possible on AWS. With Amazon EKS Anywhere, LG CNS provides solutions for customers to achieve business modernization goals beyond application modernization. – Tae Hoon Kim, VP of Cloud Business, LG CNS
Nirmata: Amazon EKS Anywhere on Bare Metal brings the proven EKS distribution to bare metal infrastructure. Now, AWS customers can get consistent Kubernetes experience in the cloud and on-premises, on virtual machines and bare metal. Nirmata supports EKS Anywhere Bare Metal and provides unified operations, governance and security for EKS and EKS Anywhere clusters independent of where and how they are deployed. As an APN Partner, we are super excited to participate in this launch and look forward to working closely with the AWS team on accelerating our customers’ cloud-native journey. – Ritesh Patel, VP Products at Nirmata
Rafay: Rafay is pleased to be partnered with AWS on EKS Anywhere Bare Metal environments. Rafay’s Kubernetes Operations Platform (KOP) is uniquely positioned to help customers deliver enterprise-grade automation, security, visibility and governance requirements. Customers planning to leverage EKS Anywhere for on-premises, bare metal environments can now operate both public cloud and on-premises based clusters easily with Rafay. – Mohan Atreya, SVP of Product and Solutions, Rayfay
Tech Mahindra: Tech Mahindra’s netOps.ai enables a fully managed hybrid telco cloud experience on AWS using EKS-Anywhere on-premises bare metal hardware through a single click. netOps.ai simplifies all operations, which are typically one of the most significant sources of expenditures for dedicated on-premises networks, by running such operations on EKS-Anywhere providing continuous integration and continuous delivery pipelines for the entire lifecycle of the Solution and a fully automated installation and maintenance. – Tulasi Ram Bora, Chief Architect for ai in Tech Mahindra
Tetrate: “Customers, while migrating their applications to the cloud and embracing containers, often choose to span their application between on-premises and cloud,” said Varun Talwar, CEO of Tetrate and founding PM of Istio. “With the launch of Amazon EKS Anywhere customers can get the symmetry of experience with their container platform in hybrid environments, while Tetrate Service Bridge built on Istio and Envoy provides end to end security, observability and consistent application networking policies with a unified management plane which spans multiple environments, clusters and tenants.” – Varun Talwar, CEO, Tetrate
Learn more about EKS Anywhere enabled partners:
- Aqua – Securing Kubernetes Everywhere with EKS Anywhere (EKS-A) Bare Metal
- Canonical – Ubuntu on AWS
- Dell (PowerFlex) – Dell PowerFlex Bare Metal with Amazon Elastic Kubernetes Service Anywhere, and We Do Mean “Anywhere!”
- Equinix – Run Your Infrastructure Where You Want
- Harness – Harness AWS EKS Anywhere Bare Metal
- Kong – Kong Enterprise for Amazon Web Services
- Lenovo – Lenovo Solution for AWS ECS-A and EKS-A on ThinkAgile VX
- LG CNS – Amazon EKS(Elastic Kubernetes Service) Anywhere with LG CNS
- LTI – Amazon EKS Anywhere
- nClouds – Amazon EKS Anywhere
- New Relic – AWS and New Relic: Better Together
- Rafay – Streamlining Kubernetes operations for Amazon EKS Anywhere for bare metal environments
- SentinelOne – Securing Amazon EKS Anywhere Bare Metal with SentinelOne Singularity
- Spectro Cloud – Extending Palette to support Amazon EKS Anywhere for bare metal Kubernetes!
- Suse Rancher – Open Source Container Management Platform
- Sysdig – Adapting Security to Amazon EKS Anywhere on Bare Metal
- Tech Mahindra – netOps.ai Powering Telco Networks on AWS
- Tetrate – Deployment in AWS Joining EKS Anywhere and EKS Cloud
- Nirmata – Nirmata Kubernetes Policy Management
- Pulumi – Cloud Engineering with Kubernetes
- Vertical Relevance – AWS Cloud Transformation
- Wipro – WiPro AWS Business Group
Creating a Kubernetes cluster on bare metal has typically taken a lot of work to automate the provisioning and lifecycle management for upgrades and high availability. With EKS Anywhere you can now create highly available clusters with automated upgrades with only two configuration files and a single command. You can use the same tools and workflow to create clusters on VMware, as well as on your local workstation, with more to come. If you’d like to try other provisioners, check out the full EKS Anywhere documentation here.
EKS Anywhere on Bare Metal has been the most requested feature from our customers to manage EKS clusters on-premises and get support directly from AWS. This allows you to run fully supported clusters on your own hardware, in your own datacenters, with all of the performance, compliance, and scalability you need. We’re excited to see how you use this new capability and love to hear your feedback directly on the EKS Anywhere GitHub repo or through your account representatives.
- If you would like to watch a live demo of a cluster creation process, check out Containers from the Couch where we have videos and demos with more information about EKS Anywhere.
- We also have a live stream planned for the week of July 6th to answer your questions and demonstrate a full cluster provisioning process.
- And be sure to subscribe to Containers from the Couch to get notified when new videos are posted!