AWS Open Source Blog

Integrating EC2 macOS workers with EKS and GitLab

At our annual re:Invent conference in December 2020 we announced an all new macOS-based Amazon Elastic Compute Cloud (Amazon EC2) instance. This new instance allows developers to build, test, and package their applications for all Apple platforms, such as macOS, iOS, iPadOS, tvOS, and watchOS. Customers have been asking us for ways to integrate their native build tools, such as Jenkins and GitLab, with this new instance type. We published a blog post in late December that walks through integrating Jenkins with the AWS Cloud Development Kit (AWS CDK).

In this post, we will walk through configuring GitLab with a mac1.metal Amazon EC2 instance using another open source infrastructure as code (IaC) automation tool, Pulumi.

Deployment overview

We’re seeing a new infrastructure automation trend emerging. Historically, developers have been using AWS CloudFormation or Terraform to automate AWS infrastructure deployments. Both of these tools have their respective strengths, but they require learning their own domain-specific language or writing “code” in YAML/JSON. I don’t know about you, but I prefer to write my automation using languages I’m already familiar with, such as Python or JavaScript/TypeScript. The trend we’re seeing is that developers are now using their programming language of choice with tools such as Pulumi or AWS Cloud Development Kit to build infrastructure.

Pulumi is an AWS partner that allows developers to build their cloud-native infrastructure using a familiar programming language. In addition to building infrastructure, Pulumi also has providers for integrating common tooling, such as Datadog or PagerDuty, via code.

For this post, we’ll use Pulumi’s native AWS resources to:

We’ll also use the Kubernetes package manager, Helm, to deploy GitLab to our Amazon EKS cluster. The entire process for getting everything set up will take about 45 minutes. Please note that this is an example and should be used as a guide for your site reliability engineering teams to develop their own production deployment.

Let’s get started!

Requirements

To deploy this environment, you will need to install Pulumi and have the AWS CLI installed and configured. Additionally, ensure that you have kubectl installed to work with your EKS cluster. Also, make sure the Kubernetes package manager, Helm, is installed.

Note: This project will be using Route 53 to host DNS for a domain we’re using with GitLab. If you don’t have an existing GitLab server, please have a domain that you want to use (this is required for TLS certificate generation).

Cloning the repo

First, clone the code from GitHub to get the project started.

bash$ git clone https://github.com/aws-samples/eks-mac1-gitlab.git
bash$ cd eks-mac1-gitlab/

Provisioning EKS, GitLab, macOS, Route 53, and an S3 Bucket

Now that we have configured the AWS CLI and generated an SSH key, we are ready to begin launching the build environment.

We will deploy multiple Pulumi stacks. Let’s take a quick look at what stacks are available in the repo.

bash$ ls
eks-vpc
mac-worker
r53-conf
s3-iam

In this example, we will use the AWS region Oregon or us-west-2. To use a different region, update the Pulumi configuration parameters in each stack.

bash$ cd eks-vpc
bash$ pulumi config set aws:region us-west-2
bash$ cd ../mac-worker
bash$ pulumi config set aws:region us-west-2
bash$ cd ../r53-conf
bash$ pulumi config set aws:region us-west-2
bash$ cd ../s3-iam
bash$ pulumi config set aws:region us-west-2

Creating an S3 Bucket for artifacts

We want to have an S3 bucket to store artifacts from our GitLab runners. We also want to lock this bucket down so there is no public access; we only want the runners themselves to be able to read and write to this bucket. This is a best practice that follows a least privilege security model.

bash$ cd s3-iam
bash$ npm i
bash$ pulumi up -y

Creating a new VPC and EKS cluster

Next, let’s deploy a new Amazon VPC and an Amazon EKS cluster. Provisioning an EKS cluster takes about 20 minutes.

bash$ cd ../eks-vpc
bash$ npm i
bash$ pulumi up -y
bash$ pulumi stack output kubeconfig > kubeconfig
bash$ export KUBECONFIG=`pwd`/kubeconfig

Next, let’s run the aws eks update-kubeconfig and aws eks get-token commands to properly set up kubeconfig, or we can configure the KUBECONFIG environment variable as we have done in the previous command.

Installing GitLab with Helm

To install GitLab, update the internet-accessible domain (that you own) and email address in the following command:

bash$ helm install gitlab gitlab/gitlab \                   
  --set global.hosts.domain=yourdomain.com \
  --set certmanager-issuer.email=admin@yourdomain.com

The installation takes about five minutes to complete, and we can check the status with the following command:

bash$ kubectl get pods
NAME                                                    READY   STATUS      RESTARTS   AGE
gitlab-cainjector-67dbdcc896-wlxbl                      1/1     Running     0          47s
gitlab-cert-manager-564fc9d7f5-ctdqw                    1/1     Running     0          46s
gitlab-gitaly-0                                         1/1     Running     0          46s
gitlab-gitlab-exporter-5c4b45cff8-hw7bx                 1/1     Running     0          47s
gitlab-gitlab-runner-6bbf8c5fcf-9dqmb                   0/1     Running     0          47s
gitlab-gitlab-shell-df87cf6c-v5rrv                      1/1     Running     0          32s
gitlab-gitlab-shell-df87cf6c-whpvt                      1/1     Running     0          47s
gitlab-issuer-1-4htcq                                   0/1     Completed   0          47s
gitlab-migrations-1-92h6b                               1/1     Running     0          47s
gitlab-minio-56667f8cb4-cglkv                           1/1     Running     0          47s
gitlab-minio-create-buckets-1-5lvf4                     0/1     Completed   0          47s
gitlab-nginx-ingress-controller-5d475855bb-df2pj        1/1     Running     0          46s
gitlab-nginx-ingress-controller-5d475855bb-hbvxj        1/1     Running     0          46s
gitlab-nginx-ingress-default-backend-658cc89589-zhmk8   1/1     Running     0          47s
gitlab-postgresql-0                                     2/2     Running     0          46s
gitlab-prometheus-server-768cd8f69-fv828                1/2     Running     0          46s
gitlab-redis-master-0                                   2/2     Running     0          46s
gitlab-registry-9ddb975d6-frg58                         1/1     Running     0          47s
gitlab-registry-9ddb975d6-tpglb                         1/1     Running     0          47s
gitlab-sidekiq-all-in-1-v1-5b57f74dc4-swstj             0/1     Init:2/3    0          46s
gitlab-task-runner-95499488b-q48db                      1/1     Running     0          47s
gitlab-webservice-default-565765b8c-8qfwf               0/2     Init:2/3    0          46s
gitlab-webservice-default-565765b8c-zgkk9               0/2     Init:2/3    0          47s

When all of the Kubernetes pods have transitioned to Running or Completed, the installation is done.

Next, we need to get the Elastic Load Balancer hostname. Copy this hostname because we will use it to configure Route 53.

bash$ kubectl get ingress/gitlab-webservice-default -ojsonpath='{.status.loadBalancer.ingress[0].hostname}'
123456790281923943823-8675309.us-west-2.elb.amazonaws.com

Configuring Route 53 to use your domain

Next, we will configure Route 53. First, we are going to create a new hosted zone for your domain. Second, we will create a Route 53 ALIAS DNS record entry that maps gitlab.yourdomain.com (or whatever your actual domain name is) to the load balancer that was previously configured via Helm.

bash$ cd ../r53-conf
bash$ npm i
bash$ pulumi config set elb_hostname 123456790281923943823-8675309.us-west-2.elb.amazonaws.com 
bash$ pulumi config set domain_name yourdomain.com
bash$ pulumi up -y

Note the outputs that Pulumi generates:

    domain_name : "yourdomain.com"
    elb_hostname: "123456790281923943823-8675309.us-west-2.elb.amazonaws.com"
    ns1         : "ns-1463.awsdns-55.org"
    ns2         : "ns-1996.awsdns-58.co.uk"
    ns3         : "ns-296.awsdns-38.com"
    ns4         : "ns-790.awsdns-32.net"

You will need to update your domain’s DNS to use Route 53’s nameservers. If your domain is hosted by GoDaddy, an example of how to update your nameservers can be found in the blog post Domain by GoDaddy, DNS by Route53.

Example screenshot showing GoDaddy nameservers and "own nameservers"

Note that when you update the nameservers and configure Route 53, DNS propagation can take up to 15 minutes.

Logging into GitLab

Now that we have updated Route 53, let’s log in to GitLab. The GitLab Helm chart has created a Kubernetes secret containing the root password to log in. Use the following command to capture the password:

bash$ kubectl get secret gitlab-gitlab-initial-root-password -ojsonpath='{.data.password}' | base64 --decode ; echo
1234567890abcdefghijklmnopqrstuvwxyz

In a web browser, navigate to https://gitlab.yourdomain.com (or whatever domain you chose). The login username is root and use the password from the kubectl get secret command output.

sign-in screen on GitLab

Now that we have logged in, we want to get the runner registration token to add Amazon EC2 workers to GitLab.

Select the wrench icon (Admin Area), found on the top navigation bar.

Select the wrench icon on the top nav bar

Next, select Runners.

Runners option in the Admin Area dashboard

Copy the registration token, which we will use for our Amazon EC2 macOS instance configuration.

The copy button for the registration token

Creating an EC2 macOS GitLab runner

We need to get the current organization name that we configured when installing Pulumi. To do this, run the following command and set a configuration parameter. We are setting this parameter because we are referencing outputs from our Amazon EKS, S3, and Route 53 stacks.

bash$ cd ../mac-worker
bash$ npm i
bash$ pulumi whoami
my-org-name
bash$ pulumi config set pulumi_org my-org-name

Next, create an SSH key in the AWS console if you don’t already have one. Name it pulumi-key, for example.

We now will use the registration token for the GitLab runner. We also will use the SSH key we created in the console in the first step to allow secure logins. An inbound EC2 security group is also automatically provisioned, which only allows port 22 inbound access.

bash$ cd ../mac-worker
bash$ pulumi config set --secret reg_token 123456789-my-token-goes-here-987654321
bash$ pulumi config set ssh_keyname pulumi-key
bash$ pulumi up -y

Note that it takes about 15 minutes to create and configure the macOS GitLab runner.

After about 15 minutes, we can log back into our GitLab instance (https://gitlab.yourdomain.com/admin/runners) and validate that the runner has successfully registered.

After about 15 minutes, you can now log back into your GitLab instance and validate that the runner has successfully registered, which is shown with "just now" text next to the new runner.

Now that we have successfully added a runner, we need to create a project to kick off any automated builds. Our GitLab project will require a .gitlab-ci.yml file that will tell the runner what to do when code has been committed to the repo.

Conclusion

Configuring GitLab in a reliable way can be challenging. Using automation tooling, such as Pulumi, can reduce the time it takes to build and deploy complex infrastructure and application environments. This post has provided a guide on how to configure GitLab with our new mac1.metal instance. If you run into any problems, feel free to open a GitHub issue and we’ll respond. Happy building!

Paul Roberts

Paul Roberts

Paul Roberts is a Strategic Solutions Architect for Amazon Web Services. When he is not working on serverless applications, DevOps, Open Source, or Artificial Intelligence, he is often found exploring the mountains near Lake Tahoe with his family.