How do I provide external access to multiple Kubernetes services in my Amazon EKS cluster?

Last updated: 2020-01-29

I want to provide external access to multiple Kubernetes services in my Amazon Elastic Kubernetes Service (Amazon EKS) cluster.

Short Description

You can use the NGINX Ingress Controller for Kubernetes to provide external access to multiple Kubernetes services in your Amazon EKS cluster.

Note: The Ingress Controller can be more efficient and cost-effective than a load balancer. Also, you might need to reserve your load balancer for sending traffic to different microservices.

Important: The Ingress Controller is not the same as the Ingress. The Ingress is a Kubernetes resource that exposes HTTP and HTTPS routes from outside the cluster to the services within the cluster. The Ingress Controller fulfills the ingress (usually with a load balancer). You can't use Ingress without an Ingress Controller.

Resolution

The following resolution uses the nginxinc/kubernetes-ingress Ingress Controller. The other Ingress Controller available for public use is the kubernetes/ingress-nginx. For more information, see Differences Between nginxinc/kubernetes-ingress and kubernetes/ingress-nginx Ingress Controllers.

Deploy the NGINX Ingress Controller for Kubernetes

1.    To download the NGINX Ingress Controller for Kubernetes, run the following command:

git clone https://github.com/nginxinc/kubernetes-ingress.git

2.    To choose the directory for deploying the Ingress Controller, run the following command:

cd kubernetes-ingress/deployments/

Note: All the commands in the following steps are run from the deployments directory.

3.    To create a dedicated namespace, service account, and TLS certificates (with a key) for the default server, run the following commands:

kubectl apply -f common/ns-and-sa.yaml
kubectl apply -f common/default-server-secret.yaml

Note: The default server returns a "Not Found" page with a 404 status code for all requests for domains where no Ingress rules are defined. You can use the self-signed certificate and key that are generated for testing purposes. It's a best practice to use your own certificate and key for production environments.

4.    To create a ConfigMap for customizing your NGINX configuration, run the following command:

kubectl apply -f common/nginx-config.yaml

Note: For example configurations, see ConfigMap and Annotations.

5.    To configure role-based access control (RBAC), create a ClusterRole, and then bind the ClusterRole to the service account from step 3.

kubectl apply -f rbac/rbac.yaml

Note: The ClusterRole gives the Ingress Controller permission to interact with your Amazon EKS cluster.

6.    To deploy the Ingress Controller, run the following commands.

Note: You can deploy an Ingress Controller with the Deployment or DaemonSet option. The Deployment option allows you to dynamically change the number of Ingress controller replicas. The Daemon set option allows you to deploy the Ingress Controller on every node or subset of nodes. Step 6 uses the Deployment option.

kubectl apply -f deployment/nginx-ingress.yaml
kubectl get pods --namespace=nginx-ingress

You receive output similar to the following:

NAME                            READY   STATUS    RESTARTS   AGE
nginx-ingress-fb4f4b44c-xmq6z   1/1     Running   0          3d7h

Your Ingress Controller can now accept requests from Ingress objects.

Access the Ingress Controller and run your application

For a Deployment ingress-controller, you can use a service object with type NodePort or LoadBalancer. The following steps use the LoadBalancer type.

1.    To apply your configuration, run the following commands:

kubectl apply -f service/loadbalancer-aws-elb.yaml
kubectl get svc --namespace=nginx-ingress

You receive output similar to the following:

NAME          TYPE         EXTERNAL-IP                                      PORT(S)
nginx-ingress LoadBalancer aaa71bxxxxx-11xxxxx10.us-east-1.elb.amazonaws.com 80:32462/TCP,443:32226/TCP

Note: Amazon EKS allocates a Classic Load Balancer in TCP mode with the PROXY protocol enabled to pass the client's information (the IP address and the port). You must pass this proxy information to the Ingress Controller.

2.    To configure NGINX to use the PROXY protocol so that you can pass proxy information to the Ingress Controller, add the following keys to the nginx-config.yaml file from step 1. See the following example:

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
  proxy-protocol: "True"
  real-ip-header: "proxy_protocol"
  set-real-ip-from: "0.0.0.0/0"

Note: The proxy information is passed to the Ingress Controller through the ConfigMap that you created earlier.

3.    To update the ConfigMap, run the following command:

kubectl apply -f common/nginx-config.yaml

4.    Set up your deployments or microservices (for example, hostname-app and apache-app). See the following examples.

Note: This step assumes that you're running two microservices (for demo purposes). The microservices are exposed internally with Kubernetes as the default type.

Example of a hostname-app-svc.yaml file for hostname-app:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hostname-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: hostname-app
  template:
    metadata:
      labels:
        app: hostname-app
    spec:
      containers:
      - name: hostname-app
        image: k8s.gcr.io/serve_hostname:1.1


---
apiVersion: v1
kind: Service
metadata:
  name: hostname-svc
spec:
  ports:
  - port: 80
    targetPort: 9376
    protocol: TCP
  selector:
    app: hostname-app

Example of an apache-app-svc.yaml file for apache-app:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: apache-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: apache-app
  template:
    metadata:
      labels:
        app: apache-app
    spec:
      containers:
      - name: apache-app
        image: httpd:latest
        ports:
        - containerPort: 80


---
apiVersion: v1
kind: Service
metadata:
  name: apache-svc
  labels:
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
  selector:
    app: apache-app

To apply your configurations, run the following commands:

kubectl apply -f hostname-app-svc.yaml
kubectl apply -f apache-app-svc.yaml

5.    Implement Ingress so that it interfaces with your services using a single load balancer provided by Ingress Controller. See the following micro-ingress.yaml example:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: micro-ingress
  namespace: default
spec:
  rules:
    - host: hostname.mydomain.com
      http:
        paths:
          - backend:
              serviceName: hostname-svc
              servicePort: 80
    - host: apache.mydomain.com
      http:
        paths:
          - backend:
              serviceName: apache-svc
              servicePort: 80

Note: For more information, see Name based virtual hosting.

6.    To apply your configuration, run the following command:

kubectl apply -f micro-ingress.yaml

Note: This Ingress resource defines rules that redirect anything for hostname.mydomain.com to hostname-svc, and anything for apache.mydomain.com to apache-svc. Any request that doesn't match the rule returns a 404 "Not Found" error message.

If you describe the Ingress, you receive a message similar to the following:

 Name:             micro-ingress
 Namespace:        default
 Address:          
 Default backend:  default-http-backend:80 (<none>)
 Rules:
  Host                   Path  Backends
  ----                   ----  --------
  hostname.mydomain.com  
                            hostname-svc:80 (192.168.2.45:9376,192.168.3.236:9376)
  apache.mydomain.com    
                            apache-svc:80 (192.168.2.47:80,192.168.3.45:80)

Test the NGINX Ingress Controller

1.    To access the DNS URL of the load balancer from the command line, run the following command:

curl -I
http://aaa71bxxxxx-11xxxxx10.us-east-1.elb.amazonaws.com/

Note: The load balancer endpoint is from the preceding Access the Ingress Controller and run your application section.

You receive output similar to the following:

HTTP/1.1 404 Not Found
Server: nginx/1.17.5
Date: Mon, 25 Nov 2019 20:50:58 GMT
Content-Type: text/html
Content-Length: 153
Connection: keep-alive

Note: The default server returns a "Not Found" page with a 404 status code for all the requests for domains where no Ingress rules are defined. The Ingress Controller, based on the defined rules, doesn't divert traffic to the specified backend service, unless the request matches with the configuration. Because the host field is configured for the Ingress object, you must supply the Host header of the request with the same hostname.

2.    Add the Host header to the request.

This is a request based on the first configured domain:

curl -I -H "Host: hostname.mydomain.com"
http://aaa25a5010daa11eaa41e121e71bd6ca-113564610.us-east-1.elb.amazonaws.com/

You receive output similar to the following:

HTTP/1.1 200 OK
Server: nginx/1.17.5
Content-Type: text/html
Content-Length: 612
Connection: keep-alive

This is a request based on the second configured domain:

curl -I -H "Host: apache.mydomain.com"
http://aaa25a5010daa11eaa41e121e71bd6ca-113564610.us-east-1.elb.amazonaws.com/

You receive output similar to the following:

HTTP/1.1 200 OK
Server: nginx/1.17.5
Content-Type: text/html
Content-Length: 45
Connection: keep-alive

After you add the Host header, the Ingress Controller can redirect traffic to the backend configured service as it matches the configuration defined in Ingress.

If you want to keep the same domain name, but divert the traffic based on the path accessed, you must add path-based routing with Ingress. See the following example:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: path-ingress
annotations:
  nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: hostname.mydomain.com
  http:
	paths:
	- path: /login
	  backend:
		serviceName: service1
		servicePort: 4200
	- path: /cart
	  backend:
		serviceName: service2
		servicePort: 8080

The preceding example returns only the 200 response when requests have hostname.mydomain.com as the host header. The requests are accessed on either the /login or /cart paths. For all other requests, 404 responses are returned. 


Did this article help you?

Anything we could improve?


Need more help?