MuleSoft Anypoint Runtime Fabric Deployment On Amazon EKS Anywhere

This post was co-written with Sparsh Agarwal, Senior Product Manager at Salesforce

Introduction

Amazon EKS Anywhere (Amazon EKS-A) takes the power of Amazon Elastic Kubernetes Service (Amazon EKS) beyond the AWS cloud and enables you to run Amazon EKS on your own infrastructure. It provides an installable software package for creating and operating Kubernetes clusters on-premises and automation tooling for cluster lifecycle support. Amazon EKS Anywhere can be installed on bare metal servers, VMware vSphere, CloudStack, Nutanix, AWS Snowball Edge and Docker. It allows you to leverage the benefits of EKS across hybrid environments, ensuring consistency and flexibility in your Kubernetes deployments. Amazon EKS Anywhere brings a consistent Amazon Web Services (AWS) management experience to your data center, building on the strengths of Amazon EKS Distro, an open-source distribution for Kubernetes used by Amazon EKS.

MuleSoft is an AWS partner that accelerates the adoption of key cloud services while allowing customers to safely unlock the data inside legacy, on-premises, or SaaS applications. MuleSoft Anypoint Runtime Fabric is a powerful runtime environment for Mule applications, providing enhanced performance, scalability, and high availability. You typically create Mule application to perform system integrations. Mule apps are configured to run in Mule Runtime. A request to a Mule app triggers Mule to encode the request and data in a Mule Event, and to pass it to either single or multiple threads. In this post, we’ll explore how to deploy Anypoint Runtime Fabric on Amazon EKS Anywhere, allowing you to run Mule applications in a Kubernetes environment across on-premises.

By combining the capabilities of Anypoint Runtime Fabric and Amazon EKS Anywhere, organizations can unlock the potential of running Mule applications in Kubernetes clusters deployed on their infrastructure. This provides a unified and scalable runtime environment for Mule applications, enabling efficient resource utilization, simplified management, and seamless integration with other services in the cloud and on-premises.

Throughout this post, we guide you through the process of setting up Amazon EKS Anywhere on VMware vSphere and deploying Anypoint Runtime Fabric on Amazon EKS Anywhere. By following the provided instructions and best practices, you can harness the power of MuleSoft and Amazon EKS Anywhere to build and manage resilient, high-performing Mule applications across hybrid environments.

Solution overview

The solution consists of setting up your VMware vSphere environment to deploy an Amazon EKS Anywhere Management cluster. This cluster will be used to deploy the workload cluster where the actual mule applications will be running. The management cluster also hosts the necessary management components for the workloads cluster such as Ingress, registry and monitoring components. In the workload cluster, the mule applications runs along with the necessary Mule runtime and runtime fabric agents.

Figure 1: MuleSoft Runtime Anytime Fabric Architecture on EKS-Anywhere

Prerequisites

• Administrative machine with machine requirements as listed in the EKS-A product page with the following tools installed:
  • AWS Command Line Interface (AWS CLI) version 2.10 or higher
  • Helm
  • eksctl
  • eks-anywhere (v0.14.0 or later)
  • kubectl
  • rtfctl
• VMware vSphere infrastructure with required resources as listed in EKS-A documentation
• Prepare vSphere for EKS Anywhere
• Amazon EKS-A Curated Packages are only available to customers with the Amazon EKS-A Enterprise Subscription. To request a free trial, use the link
• MuleSoft Anypoint Runtime Fabric License. To request a license, connect with your MuleSoft Account representative through this contact form
• Create a vSphere Cluster on VMware
  • Create an initial cluster on your vSphere environment that will be used as management cluster (to create and manage other clusters).
  • Create one or more workload clusters from the management cluster.

Walkthrough

Deploying MuleSoft Anypoint Runtime Fabric

Execute all commands mentioned in this post on the Administrative Machine.

Validate that your Kubernetes environment is ready for installation. Activation data can be obtained from the MuleSoft activation documentation page:

rtfctl validate <activation_data>

Upon successful validation, install MuleSoft Anypoint Runtime Fabric:

rtfctl install <activation_data>

You can then apply your Mule license key to this installation:

rtfctl apply mule-license $BASE64_ENCODED_LICENSE

kubectl CLI is used to access the Amazon EKS-A cluster and verifies MuleSoft deployment. For example, you can use the following command to verify MuleSoft rtf namespace and verify the deployed pods and their state:

kubectl get pods -A
NAMESPACE                              NAME                                                             READY   STATUS
capi-kubeadm-bootstrap-system          capi-kubeadm-bootstrap-controller-manager-7d6b5746b6-jw7bb       1/1     Running
capi-kubeadm-control-plane-system      capi-kubeadm-control-plane-controller-manager-574cbcd9d7-jzswd   1/1     Running
capi-system                            capi-controller-manager-5b64bc4449-mcw5c                         1/1     Running
capv-system                            capv-controller-manager-64c4954b7c-85r25                         1/1     Running
cert-manager                           cert-manager-7568b959dc-6mtnc                                    1/1     Running
eksa-packages                          cron-ecr-renew-27782220-lh8b8                                    0/1     Completed
eksa-packages                          cron-ecr-renew-27782520-z84jw                                    0/1     Completed
eksa-packages                          cron-ecr-renew-27782820-29g47                                    0/1     Completed
eksa-packages                          eks-anywhere-packages-5d6df98848-mrmbm                           1/1     Running
eksa-packages                          eksa-auth-refresher-kxnds                                        0/1     Completed
eksa-system                            eksa-controller-manager-6d5f44755c-m9bc9                         1/1     Running
kube-system                            vsphere-cloud-controller-manager-d7r5k                           1/1     Running
kube-system                            vsphere-cloud-controller-manager-fdm7m                           1/1     Running
kube-system                            vsphere-csi-node-6v497                                           3/3     Running
rtf                                    agent-5488fbd6db-m9zj8                                           2/2     Running
rtf                                    agent-upgrade-jqzs5                                              0/1     Completed
rtf                                    cluster-status-27783040-dlq8r                                    0/1     Completed
rtf                                    initial-cluster-status-64k77                                     0/1     Completed
rtf                                    mule-clusterip-service-66b4cfb785-jf6d7                          1/1     Running
rtf                                    resource-cache-7d86dd7996-tk5ql                                  2/2     Running
rtf                                    rtf-install-job-m7l7z                                            0/1     Completed

Figure 2: MuleSoft Anypoint Platform Runtime Manager console

Deploy a sample Hello world Mule application and expose it with an load balancer

We’ll use MetalLB as the external LB in this case.

Step 1: Select a range of underlay IP’s that will be used for the LoadBalancer. In the setup below the underlay IP Classless Inter-Domain Routing(CIDR) is 10.2.16.0/24:

kubectl get nodes -o wide
NAME         STATUS   ROLES                  AGE   VERSION               INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                                  KERNEL-VERSION   CONTAINER-RUNTIME
10.2.16.14   Ready    <none>                 50d   v1.23.7-eks-7709a84   10.2.16.14    10.2.16.14    Bottlerocket OS 1.9.0 (vmware-k8s-1.23)   5.10.130         containerd://1.6.6+bottlerocket
10.2.16.15   Ready    control-plane,master   11d   v1.23.7-eks-7709a84   10.2.16.15    10.2.16.15    Bottlerocket OS 1.9.0 (vmware-k8s-1.23)   5.10.130         containerd://1.6.6+bottlerocket
10.2.32.21   Ready    control-plane,master   11d   v1.23.7-eks-7709a84   10.2.32.21    10.2.32.21    Bottlerocket OS 1.9.0 (vmware-k8s-1.23)   5.10.130         containerd://1.6.6+bottlerocket
10.2.64.29   Ready    <none>                 50d   v1.23.7-eks-7709a84   10.2.64.29    10.2.64.29    Bottlerocket OS 1.9.0 (vmware-k8s-1.23)   5.10.130         containerd://1.6.6+bottlerocket

Step 2: Create a package configuration file for MetalLB. From Step 1, we see that the address range for four IP’s 10.2.16.30-10.2.16.33 are not used by any other service. In the following code snippet, we use this IP range for the MetalLB Load Balancer. This IP range should not be used by any other service.

cat << EOF > metallb.yaml
apiVersion: packages.eks.amazonaws.com/v1alpha1
kind: Package
metadata:
  creationTimestamp: null
  name: generated-metallb
  namespace: eksa-packages
spec:
  packageName: metallb
  config: |
    IPAddressPools:
      - name: default
        addresses:
          - 10.2.16.30-10.2.16.33
    L2Advertisements:
      - IPAddressPools:
        - default
EOF

Step 3: Install the EKS-A Package using EKS Anywhere CLI (don’t use kubectl and always use sudo as docker.socket permissions, which needs sudo in this environment). Before the package creation create a namespace for MetalLb controllers.

kubectl create namespace metallb-system
 
sudo eksctl anywhere create packages -f metallb.yaml —kubeconfig ~/.kube/config
-----------------------
The Amazon EKS Anywhere Curated Packages are only available to customers with the Amazon EKS Anywhere Enterprise Subscription
-----------------------
package.packages.eks.amazonaws.com/generated-metallb created

Step 4: Check if all components of MetalLB are in running state.

kubectl get all -n metallb-system
NAME                                                READY   STATUS    RESTARTS   AGE
pod/generated-metallb-controller-5dbdbdfff9-wb66j   1/1     Running   0          105s
pod/generated-metallb-speaker-9bnv4                 1/1     Running   0          105s
pod/generated-metallb-speaker-cpstb                 1/1     Running   0          105s
pod/generated-metallb-speaker-kpt2m                 1/1     Running   0          105s
pod/generated-metallb-speaker-stjdf                 1/1     Running   0          105s
 
NAME                                       DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/generated-metallb-speaker   4         4         4       4            4           kubernetes.io/os=linux   105s
 
NAME                                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/generated-metallb-controller   1/1     1            1           105s
 
NAME                                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/generated-metallb-controller-5dbdbdfff9   1         1         1       105s

Step 5: Install Nginx Ingress controller (there is enterprise version and community versions available – in this installing the community version).

helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace

Step 6: Verify that the ingress-controller service procured an external-ip (in this case its provided by MetalLB from the range provided). Once available, the curl should provide a reply as shown below.

kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.111.28.84    10.2.16.30    80:31385/TCP,443:32224/TCP   74s
ingress-nginx-controller-admission   ClusterIP      10.109.26.230   <none>        443/TCP                      74s
curl 10.2.16.30
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>

Step 7: Create an Ingress configuration manifest. Below is the sample hello world mule application that can be accessed with hostname www.helloworld.com and path /helloWorld (www.helloworld.com/helloWorld).

cat <<EOF> helloworld-sample-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: helloworld-ingress
  namespace: rtf
spec:
  ingressClassName: nginx
  rules:
    - host: www.helloworld.com
      http:
        paths:
          - pathType: Prefix
            backend:
              service:
                name: helloworld
                port:
                  number: 8081
            path: /helloWorld
EOF
kubectl apply -f helloworld-sample-ingress.yaml
kubectl get ing -A
NAMESPACE                              NAME                              CLASS       HOSTS                ADDRESS      PORTS   AGE
default                                demo-localhost                    nginx       demo.localdev.me     10.2.16.30   80      33h
f1039f97-fc61-4966-b65c-a0b96be70e54   helloworld-ingress                nginx       www.helloworld.com   10.2.16.30   80      7m53s
f1039f97-fc61-4966-b65c-a0b96be70e54   helloworld-rtf-ingress-template   nginx       rtf.example.com      10.2.16.30   80      33h

Note: Copy the IP address nginx ingress for the next step.

Step 8: In order to test the hello world application, add the static IP 10.2.16.30 to /etc/hosts file to resolve www.helloworld.com. Usually in general production setting, this IP will be added to customers IPAM like InfoBlox, etc. In this case, we set this static IP to the LoadBalancer service IP of the ingress-controller service.

cat /etc/hosts
127.0.0.1 localhost
10.2.16.30 www.helloworld.com

Step 9: Now you should be able to access the service with the path configured and the specific host in the specification.

curl www.helloworld.com/helloWorld
Hello World!

Things to know

• Operational support: There are no upfront commitments or fees to use Amazon EKS Anywhere. Customers can optionally purchase Amazon EKS Anywhere Enterprise Subscriptions for access to EKS-A Curated Packages as well as 24/7 support from AWS for all bundled tooling.
• Version support: Refer to the official Amazon EKS Anywhere and Kubernetes version support policy page.
• Pricing: Amazon EKS Anywhere Enterprise Subscription option available with EKS-A, which is required to get support for the EKS-A clusters and access to additional paid features such as Amazon EKS Anywhere Curated Packages.

Cleaning up

To delete the resources provisioned in the blog, please execute the following commands.

Kubectl delete ns metallb-system
Kubectl delete ns ingress-nginx
Kubectl delete ns rtf

Please make sure to delete the EKS Anywhere cluster using the steps mentioned in the link.

Conclusion

In this post, we showed you how to deploy MuleSoft Anypoint Runtime Fabric on Amazon EKS Anywhere to bring the power and flexibility of MuleSoft’s runtime environment to on-premises and edge locations. This combination allows organizations to build and manage scalable, resilient, and high-performing Mule applications across AWS cloud and on-premises. By following the step-by-step instructions in this blog post, you can start leveraging the capabilities of MuleSoft Anypoint Runtime Fabric on Amazon EKS Anywhere. To configure Mulesoft Anypoint Runtime Fabric on Amazon EKS, you can follow the quickstart link to deploy using CloudFormation

For more information on getting started with Amazon EKS Anywhere, check out the EKS-A workshop, EKS-A documentation, or frequently asked questions. Check out the EKS-A GitHub repository and join the community Slack channel in the Kubernetes workspace to contribute.