AWS DevOps & Developer Productivity Blog
CI/CD on Amazon EKS using AWS CodeCommit, AWS CodePipeline, AWS CodeBuild, and FluxCD
This post discusses how we can speed up the development of our Kubernetes infrastructure by using a continuous integration (CI) pipeline to build our Docker images and automatically deploy them to our Amazon Elastic Kubernetes Service (Amazon EKS) cluster using FluxCD and the GitOps philosophy as the continuous delivery (CD) element. To do so, we use an AWS managed services solution.
What is GitOps
GitOps[1] is an approach where infrastructure as code (IaC) is hosted in a git repository and follows the same merge request process as application software code. Git becomes the source of truth, and infrastructure files, are managed through a git repository. This post uses FluxCD, an open source CD system developed by Weaveworks, to keep everything that is written in our Git repositories synchronized with our Kubernetes infrastructure.
Provision the infrastructure
For this post, we created an AWS Cloud Development Kit (AWS CDK) repository written in Python, where you can deploy an infrastructure with multiple components managed by AWS and also the FluxCD installation. For a list of the prerequisites you need before deploying, see the README.md file. You can also install FluxCD yourself; for instructions, see Getting started with Flux.
Our architecture is made of a main pipeline in which developers and infrastructure administrators carry out the CI, and later rely on FluxCD as a CD system.
The pipeline has two main stages:
- Source – AWS CodeCommit stores the application repository and the Kubernetes infrastructure repository. A commit occurring in the application repository triggers the pipeline to start the Source stage, which clones the repository code.
- Build – AWS CodeBuild looks for a buildspec.yaml to build and pushes our container application to Amazon Elastic Container Registry (Amazon ECR).
The following screenshot shows our pipeline.
Our buildspec.yaml
lives in the application repository. See the following code:
After the Docker image is pushed to Amazon ECR, FluxCD retrieves the image, synchronizing the Kubernetes infrastructure based on our .yaml definition file, from the Kubernetes infrastructure repository. See the following code:
You can check the status of the sync by listing the current pods:
The key characteristic is the annotation part of the code, which automates the update when there is a new container image:
For more information, see Automated deployment of new container images.
FluxCD needs access to the Kubernetes infrastructure repository to check for any changes on this repository so they can listen for all changes and sync with Kubernetes. For example, in the following code, we change the number of replicas from two to three and commit it:
We can see the refresh on the logs of the FluxCD pod:
Conclusion
This post demonstrated how can we accelerate our container deployments on Kubernetes by building a CI/CD pipeline using self-managed services on AWS and an open-source tool like FluxCD.
[1]https://pages.awscloud.com/rs/112-TZM-766/images/Weaveworks%20GitOps_AWS_eBook_07012020.pdf
About the authors:
César Prieto Ballester is a DevOps Consultant at Amazon Web Services. He enjoys automating everything and building infrastructure using code. Apart from work, he plays electric guitar and loves riding his mountain bike.
Bruno Bardelli is a Senior DevOps Consultant at Amazon Web Services. He loves to build applications and in his free time plays video games, practices aikido, and goes on walks with his dog.