AWS Partner Network (APN) Blog
How to Use Webhooks to Automate Red Hat OpenShift App Rebuilds from AWS CodeCommit
By Mehdi Salehi, Sr. Partner Solutions Architect – AWS
By Boris Jelic, APAC OpenShift and Middleware Leader – IBM
Red Hat |
For more than a decade, Red Hat has been an Amazon Web Services (AWS) strategic partner. In 2019, IBM acquired Red Hat, which has furthered IBM’s expertise in enabling the next-generation of enterprise applications on AWS.
IBM maintains an extensive set of AWS competencies, validations, and certifications, and builds on Red Hat’s open hybrid cloud vision with vast experience in modernizing and transforming traditional applications.
One of the offerings to help joint AWS and Red Hat customers build for an open hybrid cloud future is Red Hat OpenShift Service on AWS (ROSA), a fully managed OpenShift service, jointly supported by both Red Hat and AWS.
Installation, management, maintenance, and upgrades are performed by Red Hat site reliability engineers (SRE) with joint Red Hat and Amazon support.
In this post, IBM and AWS focus on a single but common use case often observed in the field where a joint enterprise customer has made investments in the AWS Cloud and is now considering expanding and adopting ROSA service as part of the AWS portfolio.
We’ll also demonstrate how customers can automate their OpenShift builds when a change is pushed into their code repositories on AWS CodeCommit.
Overview
Source-to-Image (S2I) helps developers simplify and speed up container-based application development. For this to happen, S2I needs to be able to download the application source code from a source code repository. In this post, we’ll explore how a code update on AWS CodeCommit can initiate an application rebuild on ROSA automatically.
S2I needs a source clone secret to clone the code from CodeCommit. The following diagram shows the direction of the request and how the new code version is “pulled” from CodeCommit. To learn more about integrating ROSA with CodeCommit, see this AWS blog post.
Figure 1 – ROSA clones the code from AWS CodeCommit using secrets.
This works when we initiate a rebuild from the OpenShift side manually. In the following sections, we’ll explore how this process can be automated.
Mechanics of OpenShift Builds
We have an S2I application running in cluster1 within the rosaproject1 namespace:
We have also exposed the application at a public URL, so a Route object has been created:
Using the public URL of the route, we can reach the application from a web browser, which shows the output:
In OpenShift, a build configuration (or BuildConfig) describes a single build definition and set of triggers for when a new build is created. A BuildConfig is characterized by a build strategy and one or more sources. In our scenario, the source is a CodeCommit repository:
According to OpenShift documentation, a build is the process which is used to transform input parameters or source code into a runnable image. A BuildConfig object is the definition of the entire build process. In other words, for any BuildConfig there will be one or more builds:
If the BuildConfig is modified, a new build is created. We can also instruct the BuildConfig to initiate a new build. In both cases, the application source will be downloaded again from the repository. In the next section, we’ll explore how rebuilds can be configured to trigger automatically.
Application Rebuild Using a Webhook
To start a rebuild after an update is pushed to the code repository, a webhook should be configured. Creating and configuring a webhook for AWS CodeCommit needs a generic webhook trigger to kick off a new S2I build.
As shown in the following diagram, after a code update is pushed to CodeCommit, a notification mechanism is needed to inform ROSA that the code has changed.
Figure 2 – Trigger the build from AWS CodeCommit.
A unique BuildConfig is created for each S2I application. A BuildConfig describes a single build definition and a set of triggers for when a new build should be created. In fact, by describing a BuildConfig, we can extract the webhook URL:
- Get the name of the BuildConfig:
- Extract the webhook URL from the BuildConfig:
- Before the URL can be used as a webhook, the <secret> (which is part of the above URL) needs to be replaced with the right WebHookSecretKey. To find the WebHookSecretKey, the secret name should be determined first:
Alternatively:
- Now that the secret name is known, the secret key can be extracted:
However, this is the Base64-encoded version of the secret, so it needs to be decoded:
- The decoded secret (3c31a92a47de87e5) needs to be embedded into the webhook URL we extracted in Step 2:
- Let’s verify. By sending a POST HTTP request to this webhook (even without a payload), we should be able to trigger a build:
The success response code is a good indication, but this can also be verified by listing the builds. The following command shows we have successfully triggered a new build using the webhook, just a few seconds ago:
We can even describe the build (demo-app1-3) and get the build trigger cause. The following command shows a generic webhook—which is the POST HTTP request in step 6—has initiated this build:
The new app version will be deployed after a minute or so, showing the output:
Automation
What we just saw in the previous section is still manual. The goal is to automate the build process, such that AWS CodeCommit can invoke the webhook URL and notify the OpenShift BuildConfig automatically. There are several ways to address this, but for simplicity we are going to use AWS Lambda to invoke the hardcoded webhook URL.
Here’s the high-level sequence of the events:
- Developer sends a push request to the code repository on AWS CodeCommit.
- The update on CodeCommit triggers a Lambda function.
- Lambda function sends a POST HTTP request to the ROSA BuildConfig using the webhook URL.
- ROSA pulls a fresh copy of the application code from CodeCommit.
- A new build is created; therefore, a new version of the application becomes available.
In the following sections, we dive deeper into Steps 2 and 3 and see how they can be accomplished.
For Step 1, we assume the developer knows how to interact with Git repositories. And the last two steps (4 and 5) are automatically performed by S2I.
Define a Lambda Function with AWS CodeCommit as Event Source
An example and the steps of this scenario are in the AWS CodeCommit User Guide. As shown below, we have created a Lambda function and added a CodeCommit trigger to it.
Figure 3 – Adding AWS CodeCommit as a trigger for the Lambda function.
In the simplest form, if the Lambda function sends a POST request to the webhook URL, a new build should be created in ROSA. Figure 4 shows the configuration of a simple Lambda function that does this job using a few lines of code.
Figure 4 – The AWS Lambda code that invokes the webhook URL.
As we can see in the output, it has taken 252.87 milliseconds for the Lambda function to invoke the webhook URL and print the 200 HTTP response:
Figure 5 – The successful HTTP response after invoking the webhook URL.
We should also be able to see a new build that has been initiated by a webhook:
Finally, when the new build is completed, we can see the expected output in the front-page of the application:
Using AWS CI/CD Tools
So far, we have manually generated the webhook URL and hardcoded it into a Lambda function. But these manual steps can be performed as part of an AWS CodeBuild stage as well.
For a fully-automated mechanism, we also need to be able to log on to the ROSA cluster using credentials or a token. This challenge can be addressed by securely storing this information in AWS Secrets Manager. The continuous integration and continuous delivery (CI/CD) build stage will then be able to retrieve the credentials (or the token) from Secrets Manager programmatically.
Here is an example using the AWS Command Line Interface (CLI):
The following diagram shows the different steps of the solution—updating the code, initiating a build, retrieving the secrets, generating the webhook URL, and invoking it:
Figure 6 – The proposed solution using AWS CodePipeline.
The following sample BuildSpec shows how the AWS CodeBuild part of the solution works:
As a result, if we change application code, a new OpenShift build will be initiated through AWS CodePipeline and show the following output:
Conclusion
Full integration of Red Hat OpenShift on AWS (ROSA) with AWS CodeCommit allows customers to keep their code and CI/CD pipelines within AWS in a fully managed environment.
In this post, we demonstrated how ROSA can pull the application codes from a private Git repository on CodeCommit, and how code updates on CodeCommit can trigger a new application build on ROSA.
If you’re interested in getting hands-on experience with ROSA, this workshop is a good start. The Red Hat OpenShift Service on AWS documentation also provides more information about the service, its architecture, and how to operate it.
Red Hat – AWS Partner Spotlight
Red Hat is an AWS Competency Partner that helps customers get the most of the AWS Cloud by guiding them along their modernization journeys.
Contact Red Hat | Partner Overview | AWS Marketplace
*Already worked with Red Hat? Rate the Partner
*To review an AWS Partner, you must be a customer that has worked with them directly on a project.