Logging Windows Amazon EC2 UserData activity in Amazon CloudWatch
UserData is a set of instructions you send to an Amazon Elastic Compute Cloud (Amazon EC2) instance at creation time to customize the deployment. These are tasks that happen automatically when the instance is launching for the first time and tend to be difficult to troubleshoot.
The process requires decrypting the instance credentials, connecting to the instance and navigating to the file location, and reviewing the log. This sounds pretty simple, especially when you have to do it only a couple of times, but after that it can get pretty tedious.
In this post, I want to show you how to minimize or eliminate the need to connect to your Amazon EC2 instance by sending key event data to Amazon CloudWatch. This allows you to map out the initialization events in a simple and centralized location.
When launching an EC2 instance, you have the option to send instructions to the instance at launch time. This can be accomplished by using the Amazon Web Services (AWS) Management Console, the AWS Command Line Interface, or AWS CloudFormation. Because this blog is geared toward Windows, we will be provisioning our instance using PowerShell.
In Part 1 of this post, I break down the logging function. Part 2 shows how to use the function. Part 3 shows a PowerShell EC2 launch script that combines what we cover in Parts 1 and 2. In Part 4, I show you how to repeat the process with an AWS CloudFormation template.
Part 1. The logging function
I created this function to do the heavy lifting of writing the log entries in CloudWatch. Here is the complete function.
- We will need to pass in three parameters whenever the function is called.
- Next, we determine whether the log group exists. If it does, we reuse it; if not, we create a new one. When a new log group is created, we automatically know we need a log stream within it.
- Similar to step 2, we determine whether the log stream exists. This step creates the log stream when a new log group is created, but we need the flexibility to create multiple log streams within the same log group in case we want to separate items within the group.
- Next, we prepare the log entry by defining a new InputLogEvent object, the log Message, and the Timestamp.
- To submit multiple log entries, we need a sequence token. To obtain one, we query CloudWatch.
- If this is the first entry, there will be no $SequenceToken, and we need to account for this when submitting our log entries.
Part 2. Using the function
Now that the function is in place, we are ready to begin bootstrapping our instance. For this project, we will perform a couple of simple tasks on the instance, and we will send some data to CloudWatch as the tasks are processed.
- First, we prepare the two parameters that will remain static for the duration of our instance initialization.
Note: The $logStreamName code “*name*-” looks odd, but it allows you to automate the builds and avoid hardcoded names. I will go into more detail later in this post.
- Now we begin to bootstrap our instance. This includes the following tasks.
- Create a new user with credentials stored in AWS Secrets Manager
- Add the user to the Local Administrators and Remote Management groups
- Install a Windows feature and enable PowerShell remoting
- Complete the bootstrap
A. Create a new user
The best practice is to never send credentials in the clear, and we make sure to follow this best practice when provisioning the new user. I created a user name and password in AWS Secrets Manager. To retrieve these objects, make the following call to Secrets Manager.
Note:If you’re unfamiliar with Secrets Manager, I encourage you to visit the blog Securing passwords in AWS Quick Starts using AWS Secrets Manager.
We log the execution of the task by making a call to our logging function.
Another way to write this call is by using “splatting,” which makes the call a bit easier to read.
B. Add the user to the Local Administrators and Remote Management groups
We will now add the user to the two different groups and log the execution. We will again retrieve the user name from our Secrets Manager store and pass it to our calls.
C. Install a Windows feature and enable PowerShell remoting
Now let’s take care of the final two tasks. Install the AWS Management Console and enable PowerShell remoting.
D. Complete the bootstrap
We finish with a final CloudWatch entry that indicates the end of UserData processing.
Part 3. Putting it all together with PowerShell
Now that we have all the moving pieces ready, we can put them into a script that will provision an Amazon EC2 instance, execute the UserData tasks, and write the logs to CloudWatch. Before we begin, create an Identity and Access Management (IAM) role that has the following access.
- We begin by adding our code to a $Script variable that will be passed to the EC2 launch command. To pass the code to the variable, we make use of a Here-String.
- To send PowerShell code to our EC2 instance’s UserData pipe, we need to specify the language we are sending. This is why the code is bracketed with
- Earlier, we mentioned an odd piece of code called “*name*-“. We want logStreamName to contain the name of the instance. We could simply type the name of the instance here and be done with it.
In fact, that is what I did with the logGroupName.
However, this forces us to edit the code every time we deploy a new instance, and it negates the option to deploy a series of instances on the fly. By adding this construct (a keyword surrounded by asterisks), we can externalize the object (in this case, the “name”) and update it at execution time with a simple replace call.
The previous code replaces any occurrences of “*name*” with the contents of the $name variable (in this case, “MyWindowsInstance”).
- Now that the $Script is ready, we prepare the $UserData variable.
- Using the $name variable defined earlier, we prepare the tagging construct for the instance.
- We are now ready to combine all the elements needed to call the EC2 execution command.
One important note. To retrieve the user ID and password to be used when provisioning the new user, ensure that your instance is associated with a role that has access to Secrets Manager and is able to create CloudWatch entries. The Amazon Resource Name (ARN) for this is added in the
InstanceProfile_Arn parameter section.
Part 4. Putting it all together with a CloudFormation template
As we did in the previous section, it’s time to put what we learned together—this time with a CloudFormation template.For this example, I will create the roles required for the instance to talk to CloudWatch and Secrets Manager.
Note that, unlike the previous example, you must specify the name of the CloudWatch logs and the name of the instance ahead of time. You can, however, use parameters to externalize these options.
This template is also available in the quickstart-examples repo.
While the instance is starting up and executing the UserData section, you can head over to CloudWatch and observe the logs being generated.
The following image shows all the entries created.
In this blog post, I covered a technique to help you externalize what is happening with your Amazon EC2 instance at creation time. I showed you how to leverage AWS Secrets Manager to ensure credentials are secure and how you can make the code reusable by externalizing the elements you want to persist on a per-instance case with PowerShell. Finally, I created an AWS CloudFormation template that makes use of our UserData scripts. While this example is geared specifically toward Windows Amazon EC2 instances, you can, with a few changes, adapt this code to work with Linux builds.
I hope you find this post helpful. If you have any questions, let us know in the comments. Happy automating!