AWS Cloud Operations & Migrations Blog
Run Scripts Stored in Private or Public GitHub Repositories Using Amazon EC2 Systems Manager
By Melonia Mendonca, Software Development Engineer at Amazon Web Services
Amazon EC2 Systems Manager (SSM) lets you configure, manage and automate your AWS and on-premises resources at scale. You can perform safe and secure operations without SSH access or bastion hosts using Systems Manager Run Command, mitigate configuration drift using Systems Manager State Manager, and create an access-controlled environment with full auditing. With SSM Documents, you can author your configurations as code and enable centralized management across accounts, enforcing best practices. Systems Manger provides a number of public documents for common management scenarios, or you can create your own.
We recently announced the ability to run scripts from remote locations such as GitHub or Amazon S3. This simplifies how you automate environments by letting you use existing scripts or toolsets without having to port them over to Systems Manager or create Documents as wrapper around those scripts or tools. For information, please read our partner and product integration documentation here. For example, you can:
- Execute various types of scripts written in Python, Ruby or PowerShell. You can also run configurations such as Ansible playbooks. You can pretty much run anything on your instances as long as the software (e.g., Python 2.7 or Ansible) is installed on your instance and recognized by Shell on Linux and PowerShell on Windows
- Download scripts stored in private or public GitHub repositories, or on Amazon S3 onto your instances for execution
- Run multiple files by downloading a complete GitHub directory or an S3 bucket
Systems Manager now provides a new public Document, AWS-RunRemoteScript that runs scripts from GitHub or Amazon S3 on specified instances. It does this using the new plugin from Amazon SSM Agent, aws:downloadContent, which downloads content from locations such as public or private GitHub repositories, S3 buckets, and Documents already created on SSM. If you create your own documents, you can use the aws:downloadContent plugin, and the existing aws:runShellScript (on Linux) or aws:runPowerShellScript (on Windows) to execute the scripts.
In this blog post, I’ll show you how to run an Ansible playbook located in a public or private GitHub repository using the AWS-RunRemoteScript Document. This lets you run Ansible from an external location without requiring SSH access on your instances.
Walkthrough 1 – Run an Ansible playbook from a public GitHub repository
Pre-requisites
We will run an Ansible playbook that installs and configures NGINX from a GitHub public repository. The playbook is expressed in server.yml and this main playbook calls the nginx role.
Before you get started, ensure you have reviewed the Ansible license and then install it on your instances. You can install Ansible using Run Command with the commands below:
Amazon Linux:
Ubuntu:
Step 1: Find the AWS-RunRemoteScript document for execution
On the EC2 console, on the navigation pane at the left, under Systems Manager Services, choose Run Command. Choose Run a Command, and then select the AWS-RunRemoteScript document and the instances you want to execute this document on (whether a list of instances or tag-queries).
Step 2: Reference the Ansible playbook located on GitHub
Enter the parameters for the AWS-RunRemoteScript Document to reference the Ansible playbook.
- Source Type: Location of the script – GitHub, S3. In this case, choose GitHub.
- Source Info: Provides location information for accessing the content. In this example, since the repository is public, you only need to provide the owner, repository and the path to the playbook. The playbook needs to access the nginx directory shown in the structure that follows. So we’ll download the entire directory, which includes server.yml and the nginx directory.
- Command Line: The command needed to execute the playbook
When you are done, the console will look like this:
Step 3: Run the command
Because you referenced the top-level directory, Systems Manager downloads all the playbook YAML scripts inside the nginx directory as well as the server.yml and user-data.sh files. The server.yml is then executed based on command line parameters, which then installs and configures NGINX.
You can then view the output and see that NGINX was installed on the specified instances.
You can also perform this operation using the AWS CLI by running the following command:
Walkthrough 2 – Run Ansible playbook from private GitHub repository
Now, I’ll show you how to execute scripts from private GitHub repositories. Let’s assume that the playbook in the previous example is stored in a private GitHub repository. To access this playbook, you need to create a private access token on GitHub and store it in Amazon EC2 Systems Manager Parameter Store.
Step 1: Create your GitHub personal access token
Create a personal access token for your private GitHub repo to give Systems Manager access to the playbook. Personal API tokens are a way to provide access to systems to access information from your private GitHub repository. These tokens provide limited access to a subset of repository data as well as the ability to revoke access when needed. You can create a personal access token from information provided here and then save the token value.
Step 2: Store the tokens in Parameter Store
After creating the personal access token, go to Parameter Store on the EC2 console. On the Parameter Store page, create a parameter and add the token you created on GitHub here, in the Value text box.
If you have an AWS Key Management Service (KMS) key ID, you can add this key ID in the KMS Key ID text box. After this, choose Create Parameter. You can also perform this operation using the AWS CLI, as follows:
Step 3: Reference the Ansible playbook located on GitHub
Along with owner, repository and path, we will add “tokenInfo” that refers to the example-token secure string parameter that we just created. The reference is made using the ssm-secure prefix.
Step 3: Run the command
This command will use the personal access token to access your private GitHub repository. Everything else is the same as if you were to run the playbook from a public GitHub repository.
You can also perform this operation using the AWS CLI:
Conclusion
In this blog post, I showed you how EC2 Systems Manager is a management platform that lets you use your existing tools to manage your AWS resources and environments. I showed you how to use Systems Manager to run an Ansible playbook on your EC2 instances from a public and private GitHub repository. Using the AWS-RunRemoteScript public document or the aws:downloadContent and aws:runShellScript plugins, you can run any script such as Python, Ruby, or even PowerShell scripts or modules. In a subsequent blogpost, I’ll show you how to enable modular and reusable configurations using composite Documents.
About the Author
Melonia Mendonca is a Software Development Engineer with the Amazon EC2 Systems Manager team. She is a passionate engineer who enjoys the ability to innovate encouraged by Amazon. Outside of work, Melonia likes traveling, playing board games and trying different restaurants/cuisines.