Fully-automated enterprise-scaled provisioning of AWS Accounts via Self-Service using Jira Service Desk

With more than 5 million articles from over 7,000 brands, OTTO is one of the leading German online shopping platforms. In the future, it will open up to even more brands and partners as part of its transformation. OTTO is part of the internationally active Otto Group, with headquarters in Hamburg, and employs 6,100 people throughout Germany. In the 2020/21 financial year, OTTO generated revenues of 4.5 billion euros.

At OTTO, we faced several challenges to set up a scalable multi-account environment in AWS. We must govern several hundred AWS accounts for our product teams, all while balancing the need for agility and control. At this scale, the cloud setup and governance can be complex and time consuming, the onboarding of product teams and new environments must be accelerated, and manual steps should be reduced to a minimum. All of this is done to increase agility and decrease the rate of errors, while being fully integrated into the enterprise environment, via approval processes with SIEM-integration. SIEM, or security information and event management, is a field within computer security combining security information management and security event management products and services.

The cloud competence center at OTTO IT, also known as the Governance at Scale (GAS) team, built a serverless self-managed landing zone that is integrated in the OTTO tooling ecosystem with Jira Service Desk and Quest One Identity Manager. The solution allows product teams to provision new AWS Accounts via a self-service portal in Jira Service Desk, while the central GAS team provides pre-approved, secured, and governed AWS accounts that conform to company policies. The integration with Jira Service Desk enables the central team to track resources and configurations and maintain visibility into the compliance status, as well as provide the product teams with documentation to accelerate product team onboarding.

OTTO worked with globaldatanet to set up its Landing Zone. globaldatanet is an award-winning AWS Advanced Consulting Partner and longtime Cloud Solution Provider for OTTO, supporting the team in cloud security and GAS. Their focus on building cloud-native solutions using Serverless supported over 100 companies within 5 years to develop and innovate products and services in the cloud.

In this post, we’ll demonstrate how to automate the provisioning of AWS Accounts via self-service using Jira Service Desk, potentially reducing the provisioning time of new AWS Accounts from 4 hours to 25 minutes.

The solution workflow includes the following steps:

1. Add a new Request Type for the AWS Account creation in Jira Service Desk.
2. Use the Jira Automation feature to approve the AWS Account request and send the information to an endpoint in AWS.
3. Usage of Amazon API Gateway with AWS Lambda to process the Jira Service Desk AWS Account request payload and trigger the account vending process.
4. Using Amazon Simple Email Service (Amazon SES) to communicate the Account creation status back to Jira Service Desk and update the corresponding Jira AWS Account request.

Let’s see how this works.

Prerequisites

The following prerequisites are necessary for following along with the contents of this post:

• An existing AWS account
• Jira Service Desk Software
• Activated Email API in Jira

Solution overview

The following architecture shows the whole solution of the Account Vending Machine. In this post, we focus on the Jira Service Desk Integration, highlighted in green.

Architecture of fully-automated enterprise-scaled provisioning of AWS Accounts via SelfService using Jira Service Desk.

Architecture of fully-automated enterprise-scaled provisioning of AWS Accounts via SelfService using Jira Service Desk Connector.

Jira Service Desk Configuration

We decided against utilizing a plugin to reduce dependencies/constraints, especially with a focus on Jira Cloud. Fortunately, Jira offers everything that we need to build our own solution.

Request type

Request types let you define and organize incoming issues. Create a new Request Type dedicated to request new AWS Accounts. To make it easier for customers to find what they need, additionally create a group “Account Vending Machine” in which these request types are organized.

An example of an AWS JSD-project configuration.

The request type is based on the Issue Type “New Feature” and is provided by the OTTO-Jira-administration team. The workflow used is the standard “open”=>”in progress”=>”in validation”=>”closed”-configuration.
Note that you should restrict access to the Jira Request Type to a limited amount of resources having a business role within your enterprise to allow the request of new AWS accounts.

Request type configuration

To set the relevant metadata needed for account management and governance, the request type is configured with additional fields for AWS Account Email Address, Organizational Unit (OU), Environment, Cost Center, etc. This information is processed in the Account Vending Machine, to tag AWS resources, configure AWS Organization Service Control Policies, security guardrails, notification and alerts, and to build billing dashboards.

The fields that the OTTO account vending machine is requesting.

Especially in large environments, it’s an additional effort for Jira admins to create new fields for every use case. We circumvented this by reusing existing fields and setting a “display name”.

After setup is completed, the customers’ view on the support portal is as follows:

How it looks from the customer perspective.

As soon as the required fields have been entered and the Create button is selected, a new request is raised.
Note the field “IT-Shop Approvers”, which is very important later on. The field is based on Jira’s “Person Picker”, which lets you fetch persons from Jira’s directory, which in our case is synced against the corporate Active Directory.
Process wise, we decided against full automation, thus enabling us to briefly check the quality of the entered fields before letting the Jira Automation kick in.

Jira automation

Jira automation rules let you perform actions based on specific triggers and conditions. In our case, we want to approve the request and send the populated fields from the new AWS Account Request to trigger the account creation in AWS.
This automation rule will apply the following action, after a customer creates a new AWS account request:

1. When a GAS team member approves the request by putting it into the status “in progress”.
2. Jira Automation is triggered and sends the issue as payload to a webhook: the API gateway endpoint in AWS.

Jira Automation uses a combination of “WHEN”, “IF” and “THEN” to automate tasks.

AWS Deployment

API Gateway

The next requirement of the architecture is the API receiving our payload, for example, ”something” we could use as a Webhook URL.

The obvious solution was to use API Gateway.

It takes care of the authentication by evaluating the API-token delivered in the header and, if successful, using the Lambda integration to hand over the payload as received from Jira. See here how to set up an API Gateway with Lambda integration.

Proxy Lambda

Due to the information needed for the account-vending-machine, you can’t avoid custom fields in Jira. These fields aren’t named the same as you see within the Jira configuration. Instead, they use the format “customfield_xyz” within the payload.

To identify the fields that you need for the processing, we’d suggest populating a ticket with easily recognizable values, and then use a payload dumper such as this (or your Lambda) to check the payload for the field keys that you need.

As an alternative, you could use the “Inspect Element” function of your browser while having the JSD-ticket opened.

In our case, as shown in the ticket’s screenshot, we’re using a couple of fields, for example “E-Mail Address”, to request the account’s contact email. This field is in our OTTO-specific Jira environment identified as “customfield_12949”.
As soon as all of the necessary fields are identified, use Lambda to process the payload. Depending on the programming language that you use, you can either address them directly in Python or transform the JSON into a data object.
As soon as this is done, we suggest using Lambda for preprocessing the values, for example, making sure that there are no leading or trailing whitespaces or convert upper to lowercase.

Finally, the Lambda uses AWS SDK to trigger a new codebuild-project run, while using the variables fetched from the payload as environment variables.

Account Vending Machine Backend

The Account Vending Machine Backend consists of AWS Systems Manager Parameters, an AWS CodeBuild, a few Lambdas, and an Amazon SES. Instead of using a Lambda function for the main logic like the Account Vending Machine, we decided to use CodeBuild because we had some pitfalls using Lambdas.

Here are some problems we solved during the development:

•  If you want to create a second AWS Account during creation of a new AWS Account using AWS Organizations API, you can run only one operation at a time. A wait function was implemented to serialize the AWS Account creation requests.
• To deploy StackSet instances, you can only run one operation at a time, and have to wait until an operation is complete before starting another one. A wait function was implemented to serialize the StackSet creation requests
• Jira is using a special notation, Jira Text Formatting Notation, to have formatted comments.

As the backend is highly dependent on your setup, we’ll keep this in a black box. However, we took some routes you also could learn from:

• StackSet Deployment: We’re using the AWS Parameter Store to configure the default stacksets to deploy. We have further stacksets which are optional – teams can also order them via JSD.
• Identity Management Integration: The OTTO approach is only to use AWS-IAM-users when functional/non-human. Every human user must use their ADFS-user to authenticate and authorize, realized via IAM Identity-Provider + SAML-Response-generator on the ADFS-site.
  As the setup is fairly specific regarding your enterprises’ choice of an IDP management suite, we won’t go into details here.
•  In our case, an account has “approvers”, as requested within the mask during the order process.
• If someone wants access to the account, they must order a specific role via our IDP-management-portal, and the approvers must accept or decline the request.

Amazon SES

To send the Account creation status back to Jira Service Desk, we’re using an email which is sent through the Amazon SES Email API to the Jira instance. We verified the whole domain of our mail server to be able to send mails from different team members as the Account Vending Machine. That way, the requester knows who invoked the Account Vending Machine. In addition, we’re using DKIM to sign an email with a private key to verify that parts of the email haven’t been modified during the transit. The Amazon SES deployed is a second tools account, and we can use this SES configuration for all of our governance tools.

We decided to use the email API of Jira because we already have experience with that API, and few of our other services are already using that API to send information back to Jira tickets. In the future, we’ll move to the Jira API with all of our services.

The email body is constructed in the Account Vending Machine with relevant information about Account metadata, links to access the AWS Account and Billing Dashboard, onboarding documentation for the product teams, and deployed security guardrails. The email updates the corresponding Account Request Jira issue, so the product team can quickly start working, while the central platform team has a configuration history and tracks changes.

The Assignee of the ticket is used as the email’s sending address, thus resulting in a more personal view, as Jira uses their identity for the comment. The important thing while sending the email to Jira is using the Ticket ID as the subject, that way the comment is mapped to the ticket.

The teams are presented the following view in the Jira Service Desk AWS Account request:

25 minutes later via the Amazon SES to JIRA Email API the ticket is updated – another happy client.

Conclusion

In this post, we demonstrated a solution to integrate Jira Service Desk with an Account Vending Machine in AWS. The solution offers a self-service portal in Jira Service Desk for product teams, and the provisioning time of new AWS Accounts could be reduced from 4 hours to 25 minutes. By providing relevant information like Account metadata, as well as quick access to relevant accounts, dashboards, and documentation, the onboarding time of new teams could be reduced by the same amount. Simultaneously, the operational efforts of the central cloud platform team were reduced. In the Account request, the team tracks resources and configurations and maintains visibility into the compliance status, which improves governance and leads to an increased security posture.

Authors: