Using sidecar injection on Amazon EKS with AWS App Mesh
AWS App Mesh works on the sidecar pattern where you must add containers to extend the behavior of existing containers. Kubernetes offers mutating admission controllers that allow operations teams to automate sidecar injection. In this post, I discuss the basics of the sidecar pattern and Kubernetes admission controllers and demonstrate how the App Mesh Sidecar Injector works on Amazon Elastic Kubernetes Service (Amazon EKS) or your Kubernetes installation on Amazon EC2.
What are sidecars?
Sidecar is a microservices design pattern where a companion service runs next to your primary microservice, augmenting its abilities or intercepting resources it is utilizing. In the case of App Mesh, a sidecar container, Envoy, is used as a proxy for all ingress and egress traffic to the primary microservice. Using this sidecar pattern with Envoy we create the backbone of the service mesh, without impacting our applications.
Kubernetes can hook into actions on Kubernetes objects before the system executes them. This allows two main functions: validation and mutation. Validation works as a gatekeeper for any operation on any resource in Kubernetes, allowing you to block actions before they execute. Mutation allows you to change the specification of a Kubernetes resource before initiation. Mutation provides the functionality for App Mesh injector.
This example deploys the App Mesh sidecar injector, then tests to see if injection works.
This process involves:
- Deploying the App Mesh Sidecar Injector
- Verifying the sidecar injector deployed correctly
- Testing the sidecar injector functionality
- Creating a namespace with sidecar injection enabled
- Deploying a pod to the namespace
- Verifying the sidecar injected successfully
For this walkthrough, you should have the following prerequisites:
- An AWS account
- A running EKS cluster
- A service mesh configured in AWS App Mesh
- Working knowledge of Kubernetes basics
To learn more about the sidecar injector, see the aws-app-mesh-inject GitHub repo.
Step 1: Deploy the sidecar injector
We deploy the App Mesh Sidecar Injector to the Kubernetes cluster using preconfigured bash scripts.
- After your cluster is up and running, export your mesh name:
- Install the sidecar injector in the namespace “appmesh-inject” :
curl https://raw.githubusercontent.com/aws/aws-app-mesh-inject/master/scripts/install.sh | bash
- Verify the installation:
kubectl get pods -n appmesh-inject NAME READY STATUS RESTARTS AGE aws-app-mesh-inject-XXXXXXXX 1/1 Running 0 21s
Step 2: Testing sidecar injector
To see if the injector works, deploy a pod in a namespace labeled to enable sidecar injection.
- Create a namespace with sidecar injection enabled:
cat <<EOF | kubectl apply -f – apiVersion: v1 kind: Namespace metadata: name: appmesh-inject-test labels: appmesh.k8s.aws/sidecarInjectorWebhook: enabled EOF
Labeling the namespace appmesh.k8s.aws/sidecarInjectorWebhook: enabled enables sidecar injection on pods in that namespace.
- Deploy a pod and see if the injector works correctly.
cat <<EOF | kubectl apply -f – apiVersion: v1 kind: Pod metadata: namespace: appmesh-inject-test name: nginx labels: name: nginx annotations: appmesh.k8s.aws/virtualNode: "test" spec: containers: - name: nginx image: nginx ports: - containerPort: 80 EOF
- You can see in the pod itself if the sidecar successfully intercepted and mutated it.
kubectl get pods -n appmesh-inject-test NAME READY STATUS RESTARTS AGE nginx 2/2 Running 0 17s
The notation 2/2 indicates your NGINX pod and the sidecar running next to it.
App Mesh assigns the pod to the virtual node named <namespace>-<deployment-name>. In this example, it would be appmesh-inject-test-nginx.
Using this Virtual Node name, you can configure routing to your pod in App Mesh.
In addition to sidecar injection, you can also override some default values with labels in your pod specification.
You can specify the mesh the pod is part of by setting the mesh name in the pod annotation:
Normally, Envoy handles all ports in the pod, but you can override by using the ports annotation:
You can override the virtual node assigned to the pod by setting the virtualNode annotation:
You can also disable sidecar injector for a specific pod:
In cases where a proxy can interfere with the functionality of egress traffic you can ignore specific ports for egress:
To remove the sidecar injector from your cluster, run the following commands:
kubectl delete namespace appmesh-inject kubectl delete mutatingwebhookconfiguration aws-app-mesh-inject kubectl delete clusterrolebindings aws-app-mesh-inject-binding kubectl delete clusterrole aws-app-mesh-inject-cr
There are several widely accepted patterns you can adopt in designing your microservice architecture. The Adapter Pattern can be used to mutate disparate container outputs into one unified format, or the Ambassador Pattern, which can be used to negotiate the connection to external resources such as distributing read and write traffic to a clustered data store. The Sidecar Pattern is a popular pattern because it allows you to compose pods from containers with independent but complimentary functionality. This allows for code reuse and teams to work independently. Sidecar injection does not have to be a burden on your developers and operations by using mutating webhooks and the App Mesh Sidecar Injector to easily automate the process for injection and configuration.
You can learn more about App Mesh in our documentation or to contribute to the App Mesh Sidecar Injector please check out our GitHub repo. Please leave any comments below or reach out to me via Twitter!