AWS Management Tools Blog

AWS CloudFormation features update: support for Amazon Athena, coverage updates for S3, RDS, Kinesis and CloudWatch

As one of the most widely-used services in AWS, CloudFormation continues to expand its feature set, including adding support for Amazon Athena, two new features to protect stacks and control rollback processes, plus several new coverage updates.

CloudFormation now supports the creation of an Amazon Athena named query as a resource. Amazon Athena is a query service that makes it easy to analyze data directly from files stored in S3 using standard SQL statements. Named queries can then be executed manually from the console, CLI or programmatically via API calls.

You can now create a standard set of named queries via CloudFormation templates. To try it out, you can use some of the sample data provided by Athena, as covered in Jeff Barr’s blog post here.

(more…)

Configuring Serverless Applications Using AWS CloudFormation Custom Resources

AWS makes it easy for developers to get started developing applications in the cloud. With the extensive array of services available on AWS, developers might incorporate more than just a few components in their applications. Manually managing the resources needed for an application can become time consuming. In addition, applications usually require more than just infrastructure to function.

In this blog post, I’ll show you how to achieve additional configuration tasks such as data loading, compilation of templates, and deployment of static files using a completely serverless architecture and a single AWS CloudFormation template.

(more…)

Recover your impaired instances using EC2Rescue and Amazon EC2 Systems Manager Automation

Have you ever had an issue connecting to your Amazon EC2 Windows instance? This can be caused by any number of different reasons, but is almost always related to how the instance is configured. Unfortunately, if you can’t connect to it, you can’t fix it!

Earlier this year, AWS announced EC2Rescue for Windows, a convenient, straightforward, GUI-based troubleshooting tool that can be run on your Windows instances to troubleshoot operating system-level issues and collect advanced logs and configuration files for further analysis.

AWS listened to your feedback, and now EC2Rescue is available as a one-click, self-service, scalable automated solution for you to use via Systems Manager Automation. Starting today, there’s a new public Systems Manager Automation document, called AWSSupport-ExecuteEC2Rescue. Documentation for EC2Rescue has more details about this Automation document.

(more…)

Use AWS CloudFormation Stack Termination Protection and Rollback Triggers to Maintain Infrastructure Availability

Managing your infrastructure as code using AWS CloudFormation provides a consistent way to rapidly deliver AWS environments for your applications. As your pace of delivery increases, it’s important to ensure you have the appropriate guardrails to protect your most critical infrastructure resources.

AWS CloudFormation now includes two additional tools to help you ensure the consistent health and stability of your application environments:

  • Stack Termination Protection provides a low friction mechanism to quickly protect stacks that contain critical resources.
  • Rollback Triggers allow you to quickly revert infrastructure changes that are having a negative impact to the performance of your applications.

In this post, I’m going to examine strategies for adding these new features to your infrastructure management tool belt.

(more…)

Amazon EC2 Systems Manager as a General-Purpose DevOps Tool

This guest post was written by Andrew Rout, Engineer at Riverbed SteelCentral Office of the CTO

A long time ago, a manufacturer in Cincinnati invented Play-Doh to be used as a wallpaper cleaner. Twenty years later, an even better purpose was found for it, and kids everywhere rejoiced.

History repeats itself with Amazon EC2 Systems Manager as we discover new ways to use this service from AWS. The following walk through shows you how Run Command can be used as a DevOps tool for orchestration and for systems introspection.

The need to communicate with EC2 instances

To manage the EC2 instances that power Riverbed Technology’s SteelCentral SaaS offering, Riverbed’s DevOps team built an internal tool that allows them to perform tasks on the EC2 instances and gives them insight into the state of the environment. A UI sits on top of a backend that communicates with the EC2 instances and various other AWS services.

This internal DevOps tool allows our operations team to do the following:

  • See dashboards describing the overall health of all infrastructure components and software components of SteelCentral SaaS
  • Provision new resources as necessary
  • Troubleshoot services running on EC2 instances
  • Manage users and licensing (more…)

Automate remediation actions for Amazon EC2 notifications and beyond using EC2 Systems Manager Automation and AWS Health

You can use EC2 Systems Manager Automation to take remediation actions in response to events that may impact your AWS resources. To illustrate this concept, this post guides you through setting up automated remediation actions when an Amazon EBS backed Amazon EC2 instance is scheduled for retirement.

An instance is scheduled to be retired when AWS detects irreparable failure of the underlying hardware hosting the instance. If your instance root device is an Amazon EBS volume you can stop and start the instance at any time of your convenience before the retirement.

Amazon EC2 Systems Manager (SSM) Automation is an AWS-hosted service that simplifies common instance and system maintenance and deployment tasks at no additional cost.

(more…)

Get Disk Utilization of Your Fleet Using EC2 Systems Manager Custom Inventory Types

Amazon EC2 Systems Manager Inventory provides a centralized way to collect and query system, application, and instance metadata. Using the resource data sync feature, you can sync this metadata to Amazon S3. In Amazon S3 you can aggregate the metadata for different AWS Regions and accounts. After you sync this inventory data to Amazon S3, you can create various visuals of the data using Amazon Athena and Amazon QuickSight.

The inventory data collection policy is configured using State Manager , which in turn gets executed by aws:softwareInventory plugin in amazon-ssm-agent.

Amazon EC2 Systems Manager Inventory provides two ways to define the types of data that it collects: predefined and custom.

·       Predefined data types (with prefix AWS) are natively supported by the inventory plugin via multiple gatherers. Some examples of predefined inventory types are AWS:Application and AWS:WindowsUpdate.

·       Custom data type (with prefix Custom) is a special inventory data type that can be defined by end users. This data type provides the flexibility of collecting additional inventory data, such as server rack location of a managed instance.

In this blog, I’ll walk you through an example that shows how to use the custom inventory data type to collect disk utilization for Windows instances. We’ll use PowerShell scripts to collect disk utilization data in the Inventory. After the data is collected, we’ll use this data to get fleet-level aggregation of disk usage.

(more…)

Manage your fleet at scale using EC2 Systems Manager

This guest post was written by Michael Baker, who works as a DevOps Engineer for the Infrastructure Engineering team at Bulletproof

Introduction

The Bulletproof Group Limited has spent many years investing in system automation to assist with fleet management at scale. More recently, we have spent a significant amount of time working with Amazon EC2 Systems Manager. In this blog post, I describe how we have utilized Amazon EC2 Systems Manager on two recent customer engagements. Much has been written about the rapid change within managed services for the public cloud, but the requirement for patching operating systems is ever present. With an increasing focus on security, patching is arguably higher up our customers’ list of priorities than ever before. Our customers increasingly focus on improving the agility of their businesses. So in addition to understanding the basics, including patching, we are now designing pipelines to be both rugged and as fast as possible.

(more…)

Reducing Configuration Drift with Amazon EC2 Systems Manager State Manager and Amazon CloudWatch Events

This post was written by Anupam Shrivastava, Software Development Engineer with Amazon Web Services.

State Manager helps you automate the process of keeping your EC2 instances or virtual machines (VM) in your on-premises data center in a desired state. Some use cases for State Manager include:

In State Manager, an association is a binding between your expressed configuration in a document, and a set of targets, on a specific schedule, to ensure consistent state. As part of the recent launch, we have made it easy for customers to easily remediate their instances when they drift from a desired configuration, provide you more control on when you can reapply configurations, and also make it easy for you to track changes to State Manager associations.

In this post, I demonstrate some new State Manager features such as association names and versions, rate expressions, and Amazon CloudWatch Events integration. You start by specifying the configuration in a Systems Manager document.

(more…)

Introducing the AWS Config Rule Development Kit (RDK)

Recently, AWS Config released a Rule Development Kit (RDK) that greatly simplifies your custom rule authoring experience. The RDK is an open-source tool that helps you set up AWS Config, author rules, and then test them using a variety of AWS resource types. This allows you to focus on the development of the rule itself. The AWS Config RDK is now available for download from the aws-config-rdk GitHub repo. We follow semantic versioning, and are dedicated to maintaining backwards compatibility for each major version.

About AWS Config

AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. AWS Config continuously monitors and records your AWS resource configurations and allows you to automate the evaluation of recorded configurations against desired configurations. Rules enable you to automatically check the configuration of AWS resources recorded by AWS Config. There are 37 managed AWS Config rules by default and 34 custom rules maintained by the community in the aws-config-rules GitHub repo.

(more…)