Automate Start or Stop of Distributed SAP HANA systems using AWS Systems Manager

In recent blogs, we’ve discussed the advantages of DevOps for SAP for both build and operations and explored using AWS Chatbot to initiate Stop/Start. Today we help you fast-track the use of AWS native services for SAP operations by open-sourcing an AWS Systems Manager based solution that will Start and Stop distributed SAP HANA landscapes installed on a Linux Operating System.

You can deploy this solution in less than 5 minutes using an AWS CloudFormation template. It will facilitate an application aware stop and start of the SAP workload and EC2 instances and will notify an email address of success or failure. The only prerequisites are that the instances are set up as managed instances in AWS Systems Manager and that the tagging strategy outlined in the readme has been applied.

Before we dive into the solution, let’s examine one of the reasons why it was developed.

The Use Case

5000+ customers globally have begun their SAP on AWS journey, either migrating existing SAP workloads as-is or exploring the implementation of Suite on HANA or S/4HANA as part of their move to AWS.

While evaluating the total cost of ownership for these cloud-based SAP deployments, many have examined the flexible commercial models available from AWS and identified opportunities to reduce costs using a combination of Savings Plans, Amazon EC2 Reserved Instances, and reduced operating hours using Amazon EC2 On-Demand Pricing for systems that are not required 24×7.

For many customers, non-production SAP systems including development, test, training, and sandbox instances are not in constant use. Some or all of these systems may have low uptime requirements or a short-lived role in the project cycle, in which case on-demand pricing may be a more cost-effective option.

For example, an r5.16xlarge EC2 instance that runs less than 67 hours a week in us-west-1 is cheaper using On-Demand pricing than even our most cost-effective pricing model. See the comparative costs in the diagram below and use the AWS Pricing Calculator to perform similar comparisons for your instance type and region.

Controlling uptime is a great way to reduce costs, but it comes with the operational challenges of shutting down EC2 instances when they are not in use, for example overnight and on weekends.  It also requires the stop/start process to be application-aware in a distributed landscape. For example, an SAP application server instance cannot start until its corresponding database instance is up and running, and during shut down, stopping a database host without first cleanly stopping the database application can increase the risk of database corruption.

We often see an evolution in how customers address this challenge. In the project phases – while you become familiar with the cloud setup and the operating model – it is not unusual to see a traditional approach where your SAP Basis Administrator logs on to individual systems to execute multiple startup and shutdown commands. In addition to this being a lengthy manual process, it is prone to errors and can introduce risks if the appropriate controls are not in place.

The next iteration we often see, uses a combination of scripts scheduled in cron and/or tag-based shutdown/startup schedules. This reduces the manual steps, but still has the challenge of coordinating cross instance dependencies and can fail to minimize system uptime to only when the system is in use. Manual setup effort, scheduling changes, and visibility are other concerns listed with this approach.

Finally, we see some customers adopt off-the-shelf solutions to manage their SAP landscapes. This approach can make sense if the product provides a broad set of operational functionality and is integrated with SAP and AWS functions. In your evaluation ensure that costs relating to licensing, hosting, implementation and support are factored in.

Understanding the limitation of all these solutions we looked for an alternative. The AWS cloud-native solution we have developed can be broken down into two parts:

• SAP Aware Systems Manager Automation Documents (Runbooks). An SAP-aware and cloud-native method of starting/stopping systems that move all instance operations (including instance start) to a central control mechanism.
• Execution and Notification Frameworks. Options to remove or reduce the dependency on the technical team but ensure the necessary controls and visibility are in place.

The resulting solution has been used by customers including CHS Inc., a diversified global agribusiness cooperative, who have continued their DevOps for SAP journey since migrating to AWS. CHS leveraged this automation to shut down their non-production SAP systems after business hours, all with no human intervention. CHS quickly saw meaningful cost savings by keeping non-productions shut down during non-business hours.

The Solution

Let’s take a more detailed look at the two elements we described:

SAP Aware Systems Manager Automation Documents

AWS Systems Manager provides a unified user interface so you can view operational data from multiple AWS services and automate operational tasks across your AWS resources. For operators with a system administration background, this should be easy to configure using a combination of predefined automation playbooks, RunCommand modules which allow writing simple bash scripts, and the occasional decision step.

Each step is designed to perform a single action or step, allowing the elements to be built, chained together, and reused but also giving improved visibility and control. (This becomes a key element of the framework later on).

We chose RunCommand and bash scripts because this aligns with the “command line” use that SAP administrators would be familiar with, we also tried to minimize the configuration and input required, using queries on the host to identify what was running and to derive the parameters required for issuing commands. To tie the execution together and identify instances SSM Automation document parameters, outputs, and instance tags were used.

Once you have deployed the document you can review the Markup text descriptions to understand the steps in more detail.

As an example, the following are amongst the steps executed if the start action is selected.

The following code snippet gives an example of the bash script used to run sapcontrol commands. See Note 1763593 – Starting and stopping SAP system instances – startsap/stopsap are deprecated (requires SAP s-user)

#!/bin/bash
SAPProfile=`grep -o 'pf=[^[:space:]]*' /usr/sap/sapservices | grep _HDB`
SID=`echo ${SAPProfile} | awk -F "/" '{print $4}'`
SYSTEMNO=`echo ${SAPProfile} | awk -F "/" '{print $7}' | awk -F "_" '{print $2}' | sed 's@^[^0-9]*\([0-9]\+\).*@\1@'`

echo "Running /usr/sap/hostctrl/exe/sapcontrol -nr ${SYSTEMNO} -function GetSystemInstanceList"

/usr/sap/hostctrl/exe/sapcontrol -nr ${SYSTEMNO} -function GetSystemInstanceList
if [ $? -eq 0 ]
then
    echo "There is a system here, I am going to use start"
    /usr/sap/hostctrl/exe/sapcontrol -nr ${SYSTEMNO} -function StartWait 600 2
else
    echo "There is no service running for instance number ${SYSTEMNO} "
    exit 1
fi

/usr/sap/hostctrl/exe/sapcontrol -nr ${SYSTEMNO} -function GetProcessList
if [ $? -eq 3 ]
then
    echo "`date +%F\ %T`: The HANA Database Started Successfully"
else
    echo "The return code was $? "
    exit 1
fi

Runbook Execution and Notification Frameworks

As an added feature, consider how you could trigger the Systems Manager Runbook with the appropriate security controls and governance to align with the criticality of your system.

If granting access from an external source, ensure that you follow the best practices for IAM and run the document using roles with the least privilege.

Start by executing the automation manually to gain familiarity with the document. As a next step, explore the options for scheduling using triggers such as Amazon Eventbridge patterns that configure runbooks as a target of an EventBridge event rule. The options and details are best covered in the Systems Manager Documentation.

If schedules are not fixed, these mechanisms can be extended for on-demand and ad-hoc usage. It is possible to execute the runbook and receive notifications on the status by integrating with other services, including email and AWS Chatbot. More complex frameworks could be designed to work with external approval processes and instant messaging services.

In the CloudFormation template provided, we have adopted a simple solution using Amazon Eventbridge and Amazon SNS. Eventbridge will identify changes in the runbook status using rules which will send emails on completion or failure, including a link to the runbook execution detail in the AWS console.

Getting Started

The CloudFormation document to deploy this solution is available for use in GitHub (/aws-samples/aws-ssm-automation-for-start-stop-sap) Take a look at the README for information on how to set up your environment and the services which will be created.

What’s next

Although we hope you can use the document as is, what excites us most is the extensibility of this solution. The options for use in an SAP operational context are endless.

This document could easily be modified to

• Work with different database types
• Use PowerShell commands for Windows workloads
• Incorporate additional steps for clustered highly available central services or database setups
• Add additional pre or post steps to your start-up and shutdown procedure, such as ensuring backup completion

A similar style of document and framework could be used for

• Coordinating database and filesystem backups
• Daily checks and housekeeping with logging capability
• Resizing instances or changing storage characteristics for performance testing

As you start to explore, you’ll see that most combinations of AWS API calls and Operating System commands are possible. It is also possible to incorporate steps that rely on AWS Lambda. We look forward to seeing what our customers build, and if you need a hand with automating operational activities using native services, consider engaging the AWS Professional Services Global SAP Specialty Practice.