How on-premises users can access a SUSE HAE-protected SAP HANA instance through Amazon Route 53

Stefan Schneider is a solutions architect at Amazon Web Services (AWS).

This blog post describes the Amazon Route 53 agent, which enables on-premises users to access an SAP HANA database that is protected by SUSE Linux Enterprise High Availability Extension (SLES HAE) in the AWS Cloud. The agent provides this functionality by dispatching users through Amazon Route 53.

The agent requires setting up SUSE HAE for high availability (HA) failover, implemented through the overlay IP address agent, as described in SAP Note 2309342, SUSE Linux Enterprise High Availability Extension on AWS. (SAP notes require SAP Service Marketplace credentials.)

The Route 53 agent extends the features of SLES HAE, including the Pacemaker cluster resource management framework, beyond protecting SAP HANA databases. Using this agent jointly with the overlay IP address agent enables SAP users to use SLES HAE for all SLES-supported configurations in the AWS Cloud, including SAP Central Instances (CIs).

The Route 53 agent is currently available as an unsupported open-source tool, and the source code is provided in this blog post. AWS is currently working with SUSE to make the agent available in the upstream repository as a supported tool. You can install the agent after you set up SAP HANA in your AWS account.

How the Route 53 agent works

The current overlay IP address agent allows application servers inside a virtual private computer (VPC) to access a protected SAP HANA server in that VPC, but doesn’t provide access to on-premises applications.

This causes some inconvenience for on-premises users, because it requires applications like HANA Studio to be managed inside the VPC via RDP or a jump server. The Route 53 agent works around this restriction by using a name-based approach to allow on-premises users to connect to the VPC. The two agents operate in parallel: The overlay IP agent routes traffic from the overlay IP address to the active node. The Route 53 agent updates the name of the SAP HANA server with the current IP address.

I’ve described the internal workings of this agent in my article DNS Name Failover for Highly Available AWS Services on the Scaling Bits website. The article describes how the Route 53 hosted zone gets updated.

The Route 53 agent is independent of SAP. It also works with the SAP NetWeaver Central Instance (CI) components of SLES HAE.

Prerequisites

This article assumes that you’ve already installed the overlay IP address agent, including the SLES Pacemaker cluster. In addition, the Route 53 agent requires:

Policies for your SLES HAE cluster instances to update Route 53 records
A profile for your root user
A Route 53 private hosted zone

Adding policies

Add the following policy to your SLES HAE cluster instances, to enable them to update Route 53 A records.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1471878724000",
            "Effect": "Allow",
            "Action": [
                "route53:ChangeResourceRecordSets",
                "route53:GetChange",
                "route53:ListResourceRecordSets",
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Creating an AWS profile for your root user

The agent calls AWS CLI commands by using an AWS profile, and will use the same profile as the overlay IP agent. It may need a proxy configuration as well, as described in the Scaling Bits website.

You can choose any profile name. The agent uses cluster as the default name, so you must change any references as necessary.

Creating a Route 53 private hosted zone

The agent updates an A record in a Route 53 hosted zone. This means that you‘ll need the required infrastructure in your AWS account. For information about how to create a private hosted zone, see the AWS documentation.

You will need the following (shown here with example values):

hostedzoneid: Z22XDORCGO4P3T
fullname: suse-service.awslab.cloud.mylab.corp. (The very last dot matters!)

Installing the agent

Copy the source code listed at the end of this blog post into a text file and place it in the directory /usr/lib/ocf/resource.d/aws. This source code is available under the MIT license.

Configuring the cluster

In Pacemaker, edit the configuration of your cluster (crm configure edit) as follows:

primitive res_AWS_ROUTE53 ocf:aws:aws-vpc-route53 \
   params hostedzoneid=Z22XDORCGO4P3T ttl=10 fullname=suse-service5.awslab.cloud.mylab.corp. profile=cluster \
   op start interval=0 timeout=180 \
   op stop interval=0 timeout=180 \
   op monitor interval=300 timeout=180 \
   meta target-role=Started

Replace the following required parameters with the appropriate values:

hostedzoneid: The host zone ID of Route 53. This is the Route 53 record table.
ttl: Time to live (TTL) for the ARECORD in Route 53, in seconds. (10 is a reasonable default value.)
fullname: The full name of the service that will host the IP address; for example, suse-service.awslab.cloud.mylab.corp. (The last period is important!)
profile: The name of the AWS CLI profile of the root account.The file /root/.aws/config should have an entry which looks like this:
- [profile cluster] – where cluster represents your profile name
- region = us-east-1 (specify your current region)
- output = text (this setting is required)

Configuring AWS-specific contraints

The Route 53 agent has to operate in the same node as the SAP HANA database. You can use a constraint to force it to be in the same node.

Create a file called aws-route53-constraint.txt with the following content. Make sure that you use the same resource identifier as before.

colocation col_Route53 2000: res_AWS_ROUTE53:Started msl_SAPHana_SID_HDB00:Master
order ord_SAPHana 2000: cln_SAPHanaTopology_SID_HDB00 msl_SAPHana_SID_HDB00

In this example, the SAP SID is encoded as part of the resource name. This will differ in your configuration.

Add this file to the configuration, and run the following command as a super user. It uses the file name aws-constraint.txt:

crm configure load update aws-route53-constraint.txt

Summary

The Route 53 agent is used with the Pacemaker cluster resource management framework to extend the features of SLES HAE beyond protecting SAP HANA databases. It allows users to protect SAP Central Instances by dispatching end-users through Route 53 to find the active ABAP SAP Central Services (ASCS) server.

The agent runs as a dependent agent to the HAE SAP agents. It doesn’t require individual administration.

If you need on-premises access to your SLES HAE systems, we encourage you to install the agent, and let us know if you have any questions or feedback.

Source code for the Route 53 agent

#!/bin/bash
#
#   Copyright 2017 Amazon.com, Inc. and its affiliates. All Rights Reserved.
#   Licensed under the MIT License.
#
#  Copyright 2017 Amazon.com, Inc. and its affiliates

# Permission is hereby granted, free of charge, to any person obtaining a copy of
# this software and associated documentation files (the "Software"), to deal in
# the Software without restriction, including without limitation the rights to
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
# of the Software, and to permit persons to whom the Software is furnished to do
# so, subject to the following conditions:

# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.

# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
# OTHER DEALINGS IN THE SOFTWARE.

#
#
#
# OCF resource agent to move an IP address within a VPC in the AWS
# Written by Stefan Schneider , Martin Tegmeier (AWS)
# Based on code of Markus Guertler#
#
#
# OCF resource agent to move an IP address within a VPC in the AWS
# Written by Stefan Schneider (AWS) , Martin Tegmeier (AWS)
# Based on code of Markus Guertler (SUSE)
#
# Mar. 15, 2017, vers 1.0.2

#######################################################################
# Initialization:

: ${OCF_FUNCTIONS_DIR=${OCF_ROOT}/lib/heartbeat}
. ${OCF_FUNCTIONS_DIR}/ocf-shellfuncs

OCF_RESKEY_ttl_default=10

: ${OCF_RESKEY_ttl:=${OCF_RESKEY_ttl_default}}

#######################################################################

usage() {
	cat <<-EOT
	usage: $0 {start|stop|status|monitor|validate-all|meta-data}
	EOT
}

metadata() {
cat <<END
<?xml version="1.0"?>
<!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
<resource-agent name="aws-vpc-route53">
<version>1.0</version>
<longdesc lang="en">
Update Route53 record of Amazon Webservices EC2 by updating an entry in a
hosted zone ID table.

AWS instances will require policies which allow them to update Route53 ARecords:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1471878724000",
            "Effect": "Allow",
            "Action": [
                "route53:ChangeResourceRecordSets",
                "route53:GetChange",
                "route53:ListResourceRecordSets",
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

Example Cluster Configuration:

Use a configuration in "crm configure edit" which looks as follows. Replace
hostedzoneid, fullname and profile with the appropriate values:

primitive res_route53 ocf:heartbeat:aws-vpc-route53 \
        params hostedzoneid=Z22XDORCGO4P3T fullname=suse-service5.awslab.cloud.sap.corp. profile=cluster \
        op start interval=0 timeout=180 \
        op stop interval=0 timeout=180 \
        op monitor interval=300 timeout=180 \
        meta target-role=Started
</longdesc>
<shortdesc lang="en">Update Route53 VPC record for AWS EC2</shortdesc>
<parameters>
<parameter name="hostedzoneid" required="1">
<longdesc lang="en">
Hosted zone ID of Route 53. This is the table of
the Route 53 record.
</longdesc>
<shortdesc lang="en">AWS hosted zone ID</shortdesc>
<content type="string" default="" />
</parameter>
<parameter name="fullname" required="1">
<longdesc lang="en">
The full name of the service which will host the IP address.
Example: suse-service5.awslab.cloud.sap.corp.
Note: The trailing dot is important to Route53!
</longdesc>
<shortdesc lang="en">Full service name</shortdesc>
<content type="string" default="" />
</parameter>
<parameter name="ttl" required="0">
<longdesc lang="en">
Time to live for Route53 ARECORD
</longdesc>
<shortdesc lang="en">ARECORD TTL</shortdesc>
<content type="string" default="${OCF_RESKEY_ttl_default}" />
</parameter>
<parameter name="profile" required="1">
<longdesc lang="en">
The name of the AWS CLI profile of the root account. This
profile will have to use the "text" format for CLI output.
The file /root/.aws/config should have an entry which looks
like:

  [profile cluster]
	region = us-east-1
	output = text

"cluster" is the name which has to be used in the cluster
configuration. The region has to be the current one. The
output has to be "text".
</longdesc>
<shortdesc lang="en">AWS Profile Name</shortdesc>
<content type="string" default="" />
</parameter>
</parameters>
<actions>
<action name="start" timeout="180" />
<action name="stop" timeout="180" />
<action name="monitor" depth="0" timeout="180" interval="300" />
<action name="validate-all" timeout="5" />
<action name="meta-data" timeout="5" />
</actions>
</resource-agent>
END
}

debugger() {
	ocf_log debug "DEBUG: $1"
}

ec2ip_validate() {
	debugger "function: validate"

	# Full name
	[[ -z "$OCF_RESKEY_fullname" ]] && ocf_log error "Full name parameter not set $OCF_RESKEY_fullname!" && exit $OCF_ERR_CONFIGURED

	# Hosted Zone ID
	[[ -z "$OCF_RESKEY_hostedzoneid" ]] && ocf_log error "Hosted Zone ID parameter not set $OCF_RESKEY_hostedzoneid!" && exit $OCF_ERR_CONFIGURED

	# profile
	[[ -z "$OCF_RESKEY_profile" ]] && ocf_log error "AWS CLI profile not set $OCF_RESKEY_profile!" && exit $OCF_ERR_CONFIGURED

	# TTL
	[[ -z "$OCF_RESKEY_ttl" ]] && ocf_log error "TTL not set $OCF_RESKEY_ttl!" && exit $OCF_ERR_CONFIGURED

	debugger "Testing aws command"
	aws --version 2>&1
	if [ "$?" -gt 0 ]; then
		error "Error while executing aws command as user root! Please check if AWS CLI tools (Python flavor) are properly installed and configured." && exit $OCF_ERR_INSTALLED
	fi
	debugger "ok"

	if [ -n "$OCF_RESKEY_profile" ]; then
		AWS_PROFILE_OPT="--profile $OCF_RESKEY_profile"
	else
		AWS_PROFILE_OPT="--profile default"
	fi

	return $OCF_SUCCESS
}

ec2ip_monitor() {
	ec2ip_validate
	debugger "function: ec2ip_monitor: check Route53 record "
	IPADDRESS="$(ec2metadata aws ip | grep local-ipv4 | /usr/bin/awk '{ print $2 }')"
	ARECORD="$(aws $AWS_PROFILE_OPT route53 list-resource-record-sets --hosted-zone-id $OCF_RESKEY_hostedzoneid --query "ResourceRecordSets[?Name=='$OCF_RESKEY_fullname']" | grep RESOURCERECORDS | /usr/bin/awk '{ print $2 }' )"
	debugger "function: ec2ip_monitor: found IP address: $ARECORD ."
	if [ "${ARECORD}" == "${IPADDRESS}" ]; then
		debugger "function: ec2ip_monitor:  ARECORD $ARECORD found"
		return $OCF_SUCCESS
	else
		debugger "function: ec2ip_monitor: no ARECORD found"
		return $OCF_NOT_RUNNING
	fi

	return $OCF_SUCCESS
}

ec2ip_stop() {
	ocf_log info "EC2: Bringing down Route53 agent. (Will remove ARECORD)"
	IPADDRESS="$(ec2metadata aws ip | grep local-ipv4 | /usr/bin/awk '{ print $2 }')"
	ARECORD="$(aws $AWS_PROFILE_OPT route53 list-resource-record-sets --hosted-zone-id $OCF_RESKEY_hostedzoneid --query "ResourceRecordSets[?Name=='$OCF_RESKEY_fullname']" | grep RESOURCERECORDS | /usr/bin/awk '{ print $2 }' )"
	debugger "function: ec2ip_monitor: found IP address: $ARECORD ."
	if [ ${ARECORD} != ${IPADDRESS} ]; then
		debugger "function: ec2ip_monitor: no ARECORD found"
		return $OCF_SUCCESS
	else
		debugger "function: ec2ip_monitor:	ARECORD $ARECORD found"
		# determine IP address
		IPADDRESS="$(ec2metadata aws ip | grep local-ipv4 | /usr/bin/awk '{ print $2 }')"
		# Patch file
		debugger "function ec2ip_stop: will delete IP address to ${IPADDRESS}"
		ocf_log info "EC2: Updating Route53 $OCF_RESKEY_hostedzoneid with $IPADDRESS for $OCF_RESKEY_fullname"
		ROUTE53RECORD="/var/tmp/route53-${OCF_RESKEY_hostedzoneid}-${IPADDRESS}.json"
		echo "{ " > ${ROUTE53RECORD}
		echo "	  \"Comment\": \"Update record to reflect new IP address for a system \", " >>	${ROUTE53RECORD}
		echo "	  \"Changes\": [ " >>  ${ROUTE53RECORD}
		echo "		  { " >>  ${ROUTE53RECORD}
		echo "			  \"Action\": \"DELETE\", " >>	${ROUTE53RECORD}
		echo "			  \"ResourceRecordSet\": { " >>	 ${ROUTE53RECORD}
		echo "				  \"Name\": \"${OCF_RESKEY_fullname}\", " >>  ${ROUTE53RECORD}
		echo "				  \"Type\": \"A\", " >>	 ${ROUTE53RECORD}
		echo "				  \"TTL\": ${OCF_RESKEY_ttl}, " >>	${ROUTE53RECORD}
		echo "				  \"ResourceRecords\": [ " >>  ${ROUTE53RECORD}
		echo "					  { " >>  ${ROUTE53RECORD}
		echo "						  \"Value\": \"${IPADDRESS}\" " >>	${ROUTE53RECORD}
		echo "					  } " >>  ${ROUTE53RECORD}
		echo "				  ] " >>  ${ROUTE53RECORD}
		echo "			  } " >>  ${ROUTE53RECORD}
		echo "		  } " >>  ${ROUTE53RECORD}
		echo "	  ] " >>  ${ROUTE53RECORD}
		echo "}" >> ${ROUTE53RECORD}
		cmd="aws --profile ${OCF_RESKEY_profile} route53 change-resource-record-sets --hosted-zone-id ${OCF_RESKEY_hostedzoneid} \
		  --change-batch file://${ROUTE53RECORD} "
		debugger "function ec2ip_start: executing command: $cmd"
		CHANGEID=$($cmd | grep CHANGEINFO |	 /usr/bin/awk -F'\t' '{ print $3 }' )
		debugger "Change id: ${CHANGEID}"
		rm ${ROUTE53RECORD}
		CHANGEID=$(echo $CHANGEID |cut -d'/' -f 3 |cut -d'"' -f 1 )
		debugger "Change id: ${CHANGEID}"
		STATUS="PENDING"
		MYSECONDS=2
		while [ "$STATUS" = 'PENDING' ]; do
			sleep	${MYSECONDS}
			STATUS="$(aws --profile ${OCF_RESKEY_profile} route53 get-change --id $CHANGEID | grep CHANGEINFO |  /usr/bin/awk -F'\t' '{ print $4 }' |cut -d'"' -f 2 )"
			debugger "Waited for ${MYSECONDS} seconds and checked execution of Route 53 update status: ${STATUS} "
		done

		return $OCF_SUCCESS
	fi

	return $OCF_SUCCESS
}

ec2ip_start() {
	# determine IP address
	IPADDRESS="$(ec2metadata aws ip | grep local-ipv4 | /usr/bin/awk '{ print $2 }')"
	# Patch file
	debugger "function ec2ip_start: will update IP address to ${IPADDRESS}"
	ocf_log info "EC2: Updating Route53 $OCF_RESKEY_hostedzoneid with $IPADDRESS for $OCF_RESKEY_fullname"
	ROUTE53RECORD="/var/tmp/route53-${OCF_RESKEY_hostedzoneid}-${IPADDRESS}.json"
	echo "{ " > ${ROUTE53RECORD}
	echo "    \"Comment\": \"Update record to reflect new IP address for a system \", " >>  ${ROUTE53RECORD}
	echo "    \"Changes\": [ " >>  ${ROUTE53RECORD}
	echo "        { " >>  ${ROUTE53RECORD}
	echo "            \"Action\": \"UPSERT\", " >>  ${ROUTE53RECORD}
	echo "            \"ResourceRecordSet\": { " >>  ${ROUTE53RECORD}
	echo "                \"Name\": \"${OCF_RESKEY_fullname}\", " >>  ${ROUTE53RECORD}
	echo "                \"Type\": \"A\", " >>  ${ROUTE53RECORD}
	echo "                \"TTL\": ${OCF_RESKEY_ttl} , " >>  ${ROUTE53RECORD}
	echo "                \"ResourceRecords\": [ " >>  ${ROUTE53RECORD}
	echo "                    { " >>  ${ROUTE53RECORD}
	echo "                        \"Value\": \"${IPADDRESS}\" " >>  ${ROUTE53RECORD}
	echo "                    } " >>  ${ROUTE53RECORD}
	echo "                ] " >>  ${ROUTE53RECORD}
	echo "            } " >>  ${ROUTE53RECORD}
	echo "        } " >>  ${ROUTE53RECORD}
	echo "    ] " >>  ${ROUTE53RECORD}
	echo "}" >> ${ROUTE53RECORD}
	cmd="aws --profile ${OCF_RESKEY_profile} route53 change-resource-record-sets --hosted-zone-id ${OCF_RESKEY_hostedzoneid} \
	  --change-batch file://${ROUTE53RECORD} "
	debugger "function ec2ip_start: executing command: $cmd"
	CHANGEID=$($cmd | grep CHANGEINFO |  /usr/bin/awk -F'\t' '{ print $3 }' )
	debugger "Change id: ${CHANGEID}"
	rm ${ROUTE53RECORD}
	CHANGEID=$(echo $CHANGEID |cut -d'/' -f 3 |cut -d'"' -f 1 )
	debugger "Change id: ${CHANGEID}"
	STATUS="PENDING"
	MYSECONDS=2
	while [ "$STATUS" = 'PENDING' ]; do
		sleep  ${MYSECONDS}
		STATUS="$(aws --profile ${OCF_RESKEY_profile} route53 get-change --id $CHANGEID | grep CHANGEINFO |  /usr/bin/awk -F'\t' '{ print $4 }' |cut -d'"' -f 2 )"
		debugger "Waited for ${MYSECONDS} seconds and checked execution of Route 53 update status: ${STATUS} "
	done

	return $OCF_SUCCESS
}

###############################################################################

case $__OCF_ACTION in
	usage|help)
		usage
		exit $OCF_SUCCESS
		;;
	meta-data)
		metadata
		exit $OCF_SUCCESS
		;;
	monitor)
		ec2ip_monitor
		;;
	stop)
		ec2ip_stop
		;;
	validate-all)
		ec2ip_validate
		;;
	start)
		ec2ip_start
		;;
	*)
		usage
		exit $OCF_ERR_UNIMPLEMENTED
		;;
esac

AWS for SAP