Introducing advanced Kubernetes control plane configuration in Amazon EKS

Customers running large Amazon Elastic Kubernetes Service (Amazon EKS) clusters want to optimize pod placement strategies for higher resource utilization and lower compute costs. Teams running high-churn batch, continuous integration and continuous delivery (CI/CD), and artificial intelligence and machine learning (AI/ML) workloads want to customize resource lifecycle management settings such as event retention duration to reduce etcd storage pressure. And customers migrating from self-managed Kubernetes want to preserve scheduler settings they have tuned over the years. Previously, many of these needs meant building and maintaining your own solution, adding operational overhead and maintenance burden. Today, Amazon EKS introduces advanced Kubernetes control plane configuration. You can now set parameters on the API server, scheduler, and controller manager directly in Amazon EKS. You continue to receive the same Amazon EKS availability and performance characteristics.

In this post, we walk through the feature, review the supported parameters, and then get hands-on with two walkthroughs: first, we configure pod placement strategies using the MostAllocated scoring strategy to prefer resource utilization over the default spreading behavior. Then, we configure event retention duration to balance debugging capabilities with storage efficiency.

What you can configure

With the scheduler customization, you can configure pod placement strategies to control how nodes are scored for pod scheduling. By default, the Kubernetes scheduler uses the LeastAllocated scoring strategy, which prefers nodes with lower resource allocation to favor availability. The MostAllocated scoring strategy prefers nodes with higher existing allocation, optimizing resource utilization. By packing pods onto nodes that are already well utilized, the MostAllocated strategy runs the same workloads on fewer nodes, which reduces compute costs and leaves fewer partially idle instances in your cluster.

The controller manager configuration gives you control over how frequently the Horizontal Pod Autoscaler (HPA) evaluates scaling decisions. Shortening the sync period means your workloads scale out sooner after load increases, because the HPA controller evaluates and acts on metrics more often. On Amazon EKS Provisioned Control Plane clusters, this pairs with a recent improvement that increased HPA sync concurrency to up to 40x the default Kubernetes value (announcement). Together, the higher concurrency and shorter sync period help the HPA controller process more scaling events in parallel and react more quickly to changing workload demand, which results in faster and more responsive pod autoscaling.

For the API server, event retention duration can be configured to balance debugging capabilities with storage efficiency. Clusters with high pod churn, such as batch pipelines, CI/CD runners, and AI/ML jobs, generate large volumes of Kubernetes events that consume etcd space and compete with the objects your cluster needs to run. Shortening retention clears this short-lived diagnostic data sooner, keeping etcd smaller and list and watch requests faster, while a longer window gives your teams more history to troubleshoot with. You can also change the service node port range to align NodePort service allocation with your organization’s network and firewall policies, or to widen the default range for clusters running many NodePort services. This configuration helps during migrations. Legacy applications and the clients that call them often expect services on specific, fixed ports. When those ports fall outside the Kubernetes default range, the usual options are modifying the application or standing up an external proxy in front of it. Aligning the range with the ports your applications already use removes that work, so you can move workloads onto Amazon EKS without rewriting them or adding network components to maintain.

Supported parameters at launch

The following table summarizes the parameters available at launch. Refer to the Amazon EKS documentation for a complete list including supported values and bounds.

Note: The horizontalPodAutoscalerSyncPeriod parameter requires Amazon EKS Provisioned Control Plane, which pre-allocates control plane capacity that’s always ready for demanding workloads and sized to absorb the additional load.

These are cluster-wide settings that apply to workloads running on the cluster. The preceding table lists the initial set of supported parameters, chosen based on customer requests. If there are additional parameters you would like to configure, let us know on the AWS Containers Roadmap.

Prerequisites

The following prerequisites are necessary to complete this walkthrough:

• An AWS account.
• An Amazon EKS Auto Mode cluster running Kubernetes version 1.31 or later.
• The latest version of the AWS Command Line Interface (AWS CLI) configured on your device, or AWS CloudShell.
• kubectl configured to communicate with the cluster.
• An AWS Identity and Access Management (IAM) principal with permissions to describe, update, and deploy workloads to the Amazon EKS cluster.

Start by setting your cluster name and AWS Region:

export CLUSTER=my-eks-cluster
export AWS_REGION=us-west-2

Walkthrough 1: Optimize pod placement with the MostAllocated scoring strategy

In this walkthrough, we configure the MostAllocated scoring strategy on an existing cluster and observe how the Kubernetes scheduler places a new pod on the node with higher existing allocation instead of the one with more available resources. With advanced Kubernetes control plane configuration, you can set this directly in your Amazon EKS cluster, eliminating the operational burden of maintaining a custom scheduler.

Step 1 – Set up a cluster with uneven node utilization

This example uses an Amazon EKS Auto Mode cluster with a static capacity node pool to maintain a fixed number of nodes regardless of pod demand. Static capacity node pools are useful for workloads that require predictable capacity, reserved instances, or specific compliance requirements where you need to maintain a consistent infrastructure footprint. Create the node pool:

cat <<EOF | kubectl apply -f -
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: demo-static
spec:
  replicas: 2
  template:
    spec:
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: default
      requirements:
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: ["m5.large"]
  limits:
    nodes: 2
EOF
kubectl wait --for=condition=Ready nodes --all --timeout=180s

After both nodes are ready, deploy workloads of different sizes to create uneven allocation:

# Heavy workload on one node (consumes most of the node capacity)
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: heavy-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: heavy-app
  template:
    metadata:
      labels:
        app: heavy-app
    spec:
      nodeSelector:
        karpenter.sh/nodepool: demo-static
      containers:
        - name: pause
          image: registry.k8s.io/pause:3.9
          resources:
            requests:
              cpu: 1500m
              memory: 1Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: light-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: light-app
  template:
    metadata:
      labels:
        app: light-app
    spec:
      nodeSelector:
        karpenter.sh/nodepool: demo-static
      containers:
        - name: pause
          image: registry.k8s.io/pause:3.9
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
EOF
kubectl wait --for=condition=Ready pods --all --timeout=120s

Verify that the resource allocation on each node by using the following commands. The scheduler uses requested resources (not actual usage) to make scoring decisions:

kubectl get pods -o wide
kubectl describe nodes -l karpenter.sh/nodepool=demo-static | grep -A 5 "Allocated resources"

Node: i-0b3a5fb6f3823fd2f
Allocated resources:
  CPU Requests: 1500m (84%)
  Memory Requests: 1Gi (14%)
Node: i-08c2daa17b2fbe865
Allocated resources:
  CPU Requests: 500m (28%)
  Memory Requests: 512Mi (7%)

Node A (i-0b3a5fb6f3823fd2f) has 84% of its CPU allocated. Node B (i-08c2daa17b2fbe865) has only 28 percent. This allocation difference is what the kube-scheduler uses when scoring nodes.

Step 2 – Observe default placement (LeastAllocated)

Deploy a new pod and observe where the scheduler places it:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  nodeSelector:
    karpenter.sh/nodepool: demo-static
  containers:
    - name: pause
      image: registry.k8s.io/pause:3.9
      resources:
        requests:
          cpu: 200m
          memory: 128Mi
EOF
kubectl wait --for=condition=Ready pods --all --timeout=120s
kubectl get pods -o wide

NAME       READY   STATUS    NODE
test-pod   1/1     Running   i-08c2daa17b2fbe865

With LeastAllocated, the scheduler placed the pod on Node B (28 percent CPU, 7 percent Memory allocated), preferring the node with more available resources. Delete this test pod before continuing:

kubectl delete pod test-pod

Step 3 – Enable the MostAllocated scoring strategy

Apply the scheduler configuration to your existing cluster:

aws eks update-cluster-config \
  --name "$CLUSTER" \
  --kube-scheduler-config '{
    "nodeResourcesFit": {
      "scoringStrategy": {
        "type": "MostAllocated"
      }
    }
  }'

Note: You can optionally specify a resources array with custom weights to influence which resources matter most in scoring decisions. Supported resources include cpu, memory, nvidia.com/gpu, aws.amazon.com/neuron, and aws.amazon.com/neuroncore, with weights from 1–100.

The response returns an update that you can track:

{
  "update": {
    "id": "a1b2c3d4-5678-90ab-cdef-EXAMPLE11111",
    "status": "InProgress",
    "type": "ControlPlaneComponentConfigUpdate"
    ...
  }

Amazon EKS applies control plane configuration updates asynchronously, similar to other cluster updates. Wait for the update to complete:

aws eks wait cluster-active --name "$CLUSTER"

This command blocks until the cluster returns to Active status, typically within a few minutes.

Step 4 – Observe new placement (MostAllocated)

New scheduling decisions now use MostAllocated. Deploy another test pod and observe where it lands:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  nodeSelector:
    karpenter.sh/nodepool: demo-static
  containers:
    - name: pause
      image: registry.k8s.io/pause:3.9
      resources:
        requests:
          cpu: 200m
          memory: 128Mi
EOF
kubectl wait --for=condition=Ready pods --all --timeout=120s
kubectl get pods -o wide

NAME       READY   STATUS    NODE
test-pod   1/1     Running   i-0b3a5fb6f3823fd2f

With MostAllocated, the scheduler now prefers the Node A (84 percent CPU, 14 percent Memory allocated) that already has higher resource allocation. Pods that specify their own scheduling requirements such as node affinity, taints and tolerations, or topology spread constraints continue to have precedence, and the scheduler honors them. Over time, this packing behavior keeps lightly utilized nodes free of new workloads, allowing them to be consolidated on node pools that support consolidation. The key takeaway: you changed the cluster-wide scheduling strategy with a single API call, no custom scheduler required.

Walkthrough 2: Configure event retention duration with eventTtl

Kubernetes stores events in the etcd database to record cluster activities such as pod scheduling decisions, image pulls, health check failures, scaling actions, and more. The upstream default retains events for one hour. On clusters running high-churn workloads like large scale batch jobs, machine learning (ML) training, CI/CD pipelines, or large-scale CronJobs, thousands of events accumulate quickly. This volume of events in etcd can become significant, adding storage pressure and slowing API server list operations.

With advanced Kubernetes control plane configuration, you can tune event retention duration to balance debugging capabilities with storage efficiency. In this walkthrough, we lower eventTtl from the default 1 hour to 15 minutes.

Step 1 – Reduce eventTtl to 15 minutes

Apply the API server configuration:

aws eks update-cluster-config \
  --name "$CLUSTER" \
  --kube-api-server-config '{
    "eventTtl": "15m"
  }'

Wait for the update to complete:

aws eks wait cluster-active --name "$CLUSTER"

Step 2 – Observe the reduction

The new eventTtl setting affects events that Kubernetes creates after the configuration change takes effect. Existing events retain their original retention period and expire accordingly. To verify the shorter retention window, create a new pod and observe its events:

kubectl run ttl-test --image=registry.k8s.io/pause:3.9
kubectl get events --field-selector involvedObject.name=ttl-test

LAST SEEN   TYPE     REASON      OBJECT         MESSAGE
6s          Normal   Scheduled   pod/ttl-test   Successfully assigned default/ttl-test to i-0b3a5fb6f3823fd2f
3s          Normal   Pulled      pod/ttl-test   Container image "registry.k8s.io/pause:3.9" already present on machine
3s          Normal   Created     pod/ttl-test   Container created
3s          Normal   Started     pod/ttl-test   Container started

After about 15 minutes, query the same events:

kubectl get events --field-selector involvedObject.name=ttl-test

No resources found in default namespace.

Kubernetes garbage-collected the events after 15 minutes, confirming the new retention window is active. Events that existed before the configuration change continue to expire at their original retention period. On clusters that run thousands of batch jobs per hour, this can free substantial etcd space and improve API server response times for event-heavy queries.

When to configure event retention

A shorter eventTtl is a good fit when:

• Your cluster runs batch, CI/CD, ML training, or CronJob workloads that generate a high volume of events.
• You observe etcd storage growing toward capacity limits.
• You rely on an external observability system (such as Amazon CloudWatch Container Insights or a log aggregator) to capture events durably, and do not depend on kubectl get events for historical debugging.

Keep in mind that a shorter TTL means kubectl describe pod and kubectl get events show a narrower window. Choose a value that balances storage efficiency with your debugging workflow.

Clean up

To avoid ongoing charges, make sure to delete Amazon EKS cluster resources created in your AWS account.

kubectl delete deployment heavy-app light-app
kubectl delete pod test-pod
kubectl delete nodepool demo-static

To delete the Amazon EKS cluster, follow the instructions from the Amazon EKS documentation.

Considerations

Before you apply advanced Kubernetes control plane configuration, keep the following in mind.

• Managed through existing Amazon EKS APIs: Advanced Kubernetes control plane configuration uses the existing CreateCluster and UpdateClusterConfig APIs with new parameters. You can apply settings through the AWS Management Console, AWS CLI, or AWS CloudFormation, with support for eksctl, AWS Controllers for Kubernetes (ACK), and Terraform planned. Amazon EKS validates each configuration before applying it, and changes are recorded in AWS CloudTrail for auditability.
• Settings are cluster-wide: Each supported parameter applies to the entire cluster and the workloads running on the cluster. Pod-level intent that Kubernetes already honors (for example, a pod’s own resource requests) still apply. The cluster configuration changes the control plane’s defaults and scoring behavior, not the Kubernetes precedence rules you rely on today.
• The scheduler and node management work at different layers: The MostAllocated scoring strategy influences how the scheduler places pods onto existing feasible nodes. It does not change how Amazon EKS Auto Mode or Karpenter provisions or removes nodes. We recommend that you validate the combined behavior for your workload before updating the configuration.
• Some parameters require Provisioned Control Plane: Parameters that can impact control plane performance or resource consumption, such as horizontalPodAutoscalerSyncPeriod, are available only on clusters with a Provisioned Control Plane tier.
• HPA sync period and cluster size: For Provisioned Control Plane clusters, Amazon EKS has increased HPA sync concurrency to up to 40x the default Kubernetes value. Together with the configurable HPA reconciliation interval (10–15 seconds), these enhancements help the HPA controller process more scaling events in parallel and react more quickly to changing workload demand. On clusters with a large number of HPA objects, validate that your chosen sync period and HPA count perform as expected in your environment before applying to production.
• Kubernetes version and Region support: Advanced Kubernetes control plane configuration is supported on new and existing Amazon EKS clusters running Kubernetes 1.31 or later versions. This is available in all AWS commercial Regions, AWS GovCloud (US), AWS China Regions where Amazon EKS is available, at no additional charge. Using horizontalPodAutoscalerSyncPeriod requires Provisioned Control Plane, which is billed at the hourly rate for your scaling tier.

Conclusion

Advanced Kubernetes control plane configuration gives cluster administrators direct control over scheduling, resource management, and API server settings while continuing to receive the availability and performance characteristics of Amazon EKS. Customers no longer need to run a modified instance of the default Kubernetes scheduler or implement workarounds, reducing operational overhead while improving resource utilization and lowering costs. In this post, you optimized pod placement with MostAllocated bin-packing for better resource utilization, and tuned event retention duration to balance debugging capabilities with storage efficiency. We encourage you to start using this feature and share your feedback at AWS Containers Roadmap.

To get started:

• Review advanced Kubernetes control plane configuration in the Amazon EKS User Guide.
• Learn about Amazon EKS Provisioned Control Plane for parameters that require pre-allocated control plane capacity.
• Explore Amazon EKS Auto Mode to simplify cluster operations with automated node management.

About the authors