Implementing custom domain names with ROSA

Red Hat OpenShift Service on AWS (ROSA) is a fully managed implementation of Red Hat OpenShift. The Red Hat SRE team does all the heavy lifting of maintaining and operating an OpenShift cluster on behalf of customers, allowing them to refocus their resources where it is most beneficial to their business and customers.

ROSA provides simplified procurement and provisioning, allowing customers to get up and running faster and streamline application development by using OpenShift to improve developer productivity. During the provisioning process, ROSA builds out a simple common domain naming structure. In this blog post, I will explore the building blocks created during ROSA cluster create and how customers can implement custom domain names for their application workloads running as containers on ROSA.

The ROSA install process does not at this stage cater for the customer to define a custom domain name. Instead, a cluster name is provided.

rosa create cluster --cluster-name test --multi-az --region us-west-2 —version 4.8.2

The ROSA provisioning process will create an AWS Route 53 hosted zone and related DNS records as part of the cluster build-out. A predefined DNS namespace is used for this <clustername>.<UniqueID>.p1.openshiftapps.com. Records will be created for the OpenShift console, API, and APPS. endpoints.

In order to make use of a custom domain, we will require a few building blocks. In this blog, I will make use of AWS Route 53 domains to register a custom domain and Route 53 hosted zones for my DNS records. I will be using the latest version of the ROSA CLI and OC CLI tools. I need to create a new ingress controller within ROSA using the custom domains operator. This ingress controller will interact with an AWS Classic Load Balancer, which provides the entry point for connecting to container application workloads running within ROSA. I will also explore options for making use of a self-assigned certification within OpenShift or bringing a custom certificate from an existing certificate authority (CA).

High-level tasks:

Registering a domain
Creating a Route 53 hosted zone
Create a custom domain and ingress controller in OpenShift
Create a ROSA cluster admin
Connect to the ROSA cluster using the cluster admin

Register a domain using Route 53 domains

Route 53 domains provide a simple way to register and manage domains. The following steps will walk you through this process.

Step 1: Sign in to the AWS Management Console and search for Route 53.

Step 2: Select Registered domains on the left-hand menu.

Step 3: Select Register domain.

Step 4: Select the desired domain name and check to make sure it is not already in use.

Stop 5: Complete the contact details for the domain.

Step 6: Complete your order.

Email verification will need to be completed before the new domain is assigned. The next step is to associate SOA and NS DNS records with the domain. For this we will be using an AWS Route hosted zone.

Create a Route 53 Hosted Zone

Step 1: From the AWS Route 53 Console, select Hosted zones on the left-hand menu.

Step 2: Select Create a hosted zone.

Step 3: Complete the domain name.

Step 4: Select if this will be a public or private hosted zone.

Step 5: Select Create.

Just a note on public vs. private hosted zones—public hosted zones can be resolved via the public internet and are good for using in combination with ROSA clusters that have public-facing applications. That is, end users are able to connect to the container application workloads across the public internet. Typically these clusters would be created with rosa create cluster — public and will have an internet-facing AWS Classic Load Balancer fronting the OpenShift ingress router.

Private hosted zones are linked to an AWS VPC and can only be resolved from within that VPC. Additional attachments to other AWS accounts and VPCs can be made. Private hosted zones are a good combination for ROSA clusters with private workloads such as clusters created via rosa create cluster — private or rosa create cluster —private-link.

Create the custom domain and ingress controller in OpenShift

I have already downloaded, installed, and configured the AWS CLI and ROSA CLI on an AWS Cloud 9 developer IDE.

I will be using this to connect to a ROSA cluster to configure the custom domain. For more information about this and provisioning a ROSA cluster, see the blog post What’s new with Red Hat OpenShift Service on AWS.

Create a ROSA cluster admin

rosa create admin --cluster=my-rosa-cluster

You should see something similar to the following:

W: It is recommended to add an identity provider to login to this cluster. See 'rosa create idp --help' for more information.
I: Admin account has been added to cluster 'my-rosa-cluster'. It may take up to a minute for the account to become active.
I: To login, run the following command:
oc login https://api.my-rosa-cluster.abcd.p1.openshiftapps.com:6443 \
--username cluster-admin \
--password FWGYL-2mkJI-00000-00000

Connect to the ROSA cluster using the cluster admin

oc login https://api.my-rosa-cluster.abcd.p1.openshiftapps.com:6443 \
        --username cluster-admin \
        --password FWGYL-2mkJI-00000-00000

First, let’s look at creating a custom domain for a private cluster using a self-signed certificate. Afterward, we will do the same for a public cluster along with importing a custom certificate. Please note, using a self-signed signed certificate will likely result in application consumers getting certificate trust warnings.

Create a nikschpri-custom-domain.yaml file with the following content:

apiVersion: managed.openshift.io/v1alpha1
kind: CustomDomain
metadata:
  name: niksch
spec:
  domain: "niksch.io"
  certificate:
    name: niksch.io
    namespace: niksch 
  scope: "Internal"

Note in the above the scope is set to internal. This means OpenShift will create an Internal Classic Load Balancer for this endpoint.

Next, I will create a custom resources using this YAML template.

oc apply -f nikschpri-custom-domain.yaml

Now let’s look at that again but from a public context.

First, we will import an existing certificate for SSL.

I will create a new TLS secret from a private key and a public certificate, where fullchain.pem and privkey.pem are the public/private wildcard certs.

oc create secret tls niksch-tls --cert=fullchain.pem --key=privkey.pem -n my-project

Create a nikschpub-custom-domain.yaml file with the following content:

apiVersion: managed.openshift.io/v1alpha1
kind: CustomDomain
metadata:
  name: niksch
spec:
  domain: "niksch.io"
  certificate:
    name: niksch-tls
    namespace: niksch 
  scope: "External"

Two differences: first, the scope is external. This will create an internet-facing AWS Classic Load Balancer. Please note, this is not supported on ROSA PrivateLink clusters! For more information regarding ROSA PrivateLink, refer to Red Hat OpenShift Service on AWS: private clusters with AWS PrivateLink.

Next, I will create a custom resources using this YAML template.

oc apply -f nikschpub-custom-domain.yaml

I can now use oc get domains to return a list of any custom domains as well as the related endpoint address.

oc get customdomains
NAME     ENDPOINT                                               DOMAIN             STATUS
niksch   xxrywp.acme.my-rosa-cluster.abcd.p1.openshiftapps.com  *.apps.niksch.io   Ready

Next, I will add a DNS record for the APPs API endpoint in my Route 53 hosted zone.

The proof of the pudding is in the eating, so I will deploy an application and expose it.

$ oc new-app --docker-image=docker.io/openshift/hello-openshift -n my-project

$ oc create route edge --service=hello-openshift hello-openshift-tls --hostname hello-openshift-tls-my-project.apps.acme.io -n my-project

$ oc get route -n my-project

$ curl https://hello-openshift-tls-my-project.apps.niksch.io
Hello OpenShift!

At this stage, custom domains can only be set for the APPs endpoint in managed OpenShift. The API and Console domains cannot be customized. OpenShift administrators, DevOps teams, and CI/CD pipeline automation toolchains will need to be able to resolve and connect to the <UniqueID>.p1.openshiftapps.com namespace automatically created during the ROSA deployment.

In this post, I ran through registering a domain using AWS registered domains, created a Route 53 hosted zone, and configured the Red Hat OpenShift Service on AWS to make use of a custom domain.

Containers