Why can't my AWS Load Balancer Controller find my subnet in Amazon EKS?

Last updated: 2023-01-12

My AWS Load Balancer Controller can't find my subnet in Amazon Elastic Kubernetes Service (Amazon EKS).

Short description

If your AWS Load Balancer Controller can't find your subnet in Amazon EKS, then you receive an error. To troubleshoot your error, complete the relevant steps for the error message that you receive.

The following error occurs when your account's AWS Identity and Access Management (IAM) role for the AWS Load Balancer Controller doesn't have the required permissions:

{"level":"error","ts":1621443417.9175518,"logger":"controller","msg":"Reconciler error","controller":"ingress","name":" ingress-2048","namespace":" game-2048","error":"couldn't auto-discover subnets: UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: 72ee57ae-f804-4f81-b069-8b04114b67b0"}

To resolve the preceding error, complete the steps in the Resolve the permission denied error section.

-or-

The following error occurs when your AWS Load Balancer Controller can't discover at least one subnet:

{"level":"error","ts":1608229710.3212903,"logger":"controller","msg":"Reconciler error","controller":"ingress","name":"ingress-2048","namespace":"game-2048","error":"couldn't auto-discover subnets: unable to discover at least one subnet"}

To resolve the preceding error, complete the steps in the Resolve the single subnet discovery error section.

-or-

The following errors occur when your AWS Load Balancer Controller can't discover two or more qualified subnets:

"msg"="Reconciler error" "error"="failed to build LoadBalancer configuration due to failed to resolve 2 qualified subnet with at least 8 free IP Addresses for ALB
{"level":"error","ts":1606329481.2930484,"logger":"controller","msg":"Reconciler error","controller":"ingress","name":"reciter-ing","namespace":"reciter","error":"InvalidSubnet: Not enough IP space available in subnet-xxxxxxxxxxxxxx. ELB requires at least 8 free IP addresses in each subnet.\n\tstatus code: 400, request id: 2a37780c-f411-xxxxx-xxxxx-xxxxxxxxx"}

To resolve the preceding errors, complete the steps in the Resolve multiple subnet discovery errors section.

Resolution

Resolve the permission denied error

1.    Enter the following command to verify that your service account is associated with the AWS Load Balancer Controller:

$ kubectl get deploy aws-load-balancer-controller -n kube-system -o yaml | grep -i serviceAccount

You receive an output similar to the following:

serviceAccount: aws-load-balancer-controller
serviceAccountName: aws-load-balancer-controller

Note: If your deployment is deployed in a different namespace, then replace -n kube-system with the appropriate namespace.

2.    Verify which IAM role is attached to the service account that's associated with the AWS Load Balancer Controller:

$ kubectl describe sa aws-load-balancer-controller -n kube-system | grep role-arn

You receive an output similar to the following:

annotations: eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxxxx:role/eksctl-cluster18-addon-iamserviceaccount-kub-Role1-xxxxxxxxxxxxx

3.    Grant ec2:DescribeAvailabilityZones permissions to the IAM role that you identify in step 2.

Resolve the single subnet discovery error

1.    Add the appropriate tags on your subnets to allow the AWS Load Balancer Ingress Controller to create a load balancer using auto-discovery.

The following is an example of private subnets tags:

kubernetes.io/role/internal-elb                Set to 1 or empty tag value for internal load balancers

The following is an example of private subnets tags:

kubernetes.io/role/elb                         Set to 1 or empty tag value for internet-facing load balancers

Note: You can manually assign subnets to your load balancer using the alb.ingress.kubernetes.io/subnets annotation. For more information, see Ingress annotations on the AWS Load Balancer Controller website.

The following is an example of a subnet with the correct tags for a cluster with an internal load balancer (private subnet):

kubernetes.io/role/internal-elb          1

The following is an example of a subnet with the correct tags for a cluster with a public load balancer (public subnet):

kubernetes.io/role/elb

2.    Tag your subnets with the appropriate format.

Key: kubernetes.io/cluster/your-cluster-name

Value: shared or owned

Important: If you're using the AWS Load Balancer Controller version v2.1.1 or earlier, then you must tag your subnets in the preceding format. Tagging is optional for versions 2.1.2 or later. It's a best practice to tag a subnet if any of the following is true:

  • You have multiple clusters that are running in the same VPC.
  • You have multiple AWS services that share subnets in a VPC.
  • You want more control over where load balancers are provisioned for each cluster.

Resolve multiple subnet discovery errors

1.    Confirm that you have at least two subnets in two different Availability Zones. This is a requirement for creating an Application Load Balancer.

Note: You can create a Network Load Balancer with a single subnet.

2.    For each subnet, specify a CIDR block with at least a /27 bitmask (for example: 10.0.0.0/27) and at least eight free IP addresses.

3.    Confirm that the tags on the subnets are formatted correctly. For example, the tags can't have any leading or trailing spaces.


Did this article help?


Do you need billing or technical support?