Tag: config


Automating the Deployment of AWS Config with the AWS SDK for PHP

by Jonathan Eskew | on | in PHP | Permalink | Comments |  Share

My colleague Joseph Fontes, an AWS Solutions Architect, wrote the guest post below to discuss automation strategies for AWS Config.


There are times when you need to automate the deployment of services either in your account or within external accounts.  When I recently had to enable AWS Config support in remote accounts, I approached this task the way many others do….by opening the AWS SDK for PHP Reference Guide!

To complete this task, we will need three AWS Config methods: putConfigurationRecorder(), putDeliveryChannel(), and startConfigurationRecorder().  Before making the call to putDeliveryChannel(), we need to create our Amazon S3 bucket destination and identify an AWS SNS topic. 

Let’s instantiate the clients we will need for this exercise.  The client creation will be slightly different for those familiar with creating clients in version 2 of the AWS SDK for PHP.

$s3Client = $sdk->createS3();
$iamClient = $sdk->createIam();
$confClient = $sdk->createConfigService();
$snsClient = $sdk->createSns();

Let’s next create an S3 bucket and destination for our AWS Config logs.  Remember that S3 buckets must be globally unique. We cannot simply use a name like “logs.”  For this reason, our naming convention will use our account number:

$accountID = "XXXXXXXXXXXX";
$s3Bucket = $accountID."-config";
$role = "AWS-Config-Role-IAM";

$s3Res = $s3Client->listBuckets([]);

$s3ResA = $s3Res->toArray();

if(bucketExists($sdk,$s3Bucket,$s3ResA) == 0) {
    $s3Data = [
        'ACL' => 'private',
        'Bucket' => $s3Bucket, 
        'LocationConstraint' => $region
    ];
    $s3Res = $s3Client->createBucket($s3Data);
    $s3ResA = $s3Res->toArray();
    print_r($s3ResA);
    print "Waiting for bucket to become available...";
    $s3Client->waitUntil('BucketExists', [
        'Bucket' => $s3Bucket
    ]);
}

In the preceding example, I have written a function to test if the bucket exists.  This is a completely optional step. The full code will be made available for download.

The call for createBucket() with Amazon S3 followed by the waitUntil() method delays script execution until the S3 bucket is available for use. 

We now need to create the IAM role AWS Config uses to access the S3 bucket.  We need an assume role policy to reference during the call.  I have created a text file, policy.json, with the following contents:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "config.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Create a local copy of the policy.json file.

Next, we need to create an additional policy file that gives the AWS Config service permissions to perform required tasks in your account. Download the following file and place it in your running directory:

config-policy.json

We can now create the IAM role for the AWS Config service:

$config_topic = "ConfigTopicSNS";
$policy_info = file_get_contents('config-policy.json');
$replace_sns = ":".$AccountID.":".$config_topic;
$replS3Bucket = "XXXXXXXXXXXXXXXXXXXXXXXXXXX";
$replSNS = "YYYYYYYYYYYYYYYYYYYYYYYYYYYYY";
$replS3Bucket2 = "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ";
$policy_info = str_replace($replS3Bucket,$s3Bucket,$policy_info);
$policy_info = str_replace($replSNS,$replace_sns,$policy_info);
$policy_info = str_replace($replS3Bucket2,$s3Bucket,$policy_info);

$iamData = [
    'RoleName' => $role,
    'AssumeRolePolicyDocument' => file_get_contents('policy.json')
];

$iamRes = $iamClient->createRole($iamData);
$iamResA = $iamRes->toArray();

$roleARN = $iamResA['Role']['Arn'];
$iamData = [
    'PolicyDocument' => $policy_info,
    'PolicyName' => 'NEWConfigPolicy',
    'Description' => "config policy via sdk"
];
$iamRes = $iamClient->createPolicy($iamData);
$iamResA = $iamRes->toArray();

$confPolicyArn = $iamResA['Policy']['Arn'];

$iamData = [
    'PolicyArn' => $ConfPolicyArn,
    'RoleName' => $Role
];
$iamRes = $iamClient->attachRolePolicy($iamData);
$iamResA = $iamRes->toArray();

This portion imports the trust policy defined by the local file, policy.json, along with the permissions policy identified by the local file, config-policy.json.  The permissions policy is modified upon read to ensure proper identifiers are used in the script.

Let’s create the SNS topic.

$snsTopic = 'ConfigTopicSNS';

$snsData = ['Name' => $config_topic];
$snsRes = $snsClient->createTopic($snsData);
$snsResA = $snsRes->toArray();

$snsTopicARN = $snsResA['TopicArn'];

We now have to call the putConfigurationRecorder() method.  This creates a new configuration recorder that will identify the changes we want to record.  In this example, we want to record all changes.  Readers can be more prescriptive in their recordings by identifying resources types.  You’ll find a list of supported resource types here:

http://docs.aws.amazon.com/config/latest/developerguide/resource-config-reference.html#supported-resources

$confData = [
    'ConfigurationRecorder' => [
        'name' => 'default',
        'recordingGroup' => [
            'allSupported' => true,
        ],
        'roleARN' => $roleARN,
    ],
];

$confClient->putConfigurationRecorder($confData);

Now that we know what we are going to record, we have to identify where we will send the recordings.  The following shows the putDeliveryChannel() method, which needs the S3 bucket (created earlier) to store the recordings and the SNS topic, which will alert to configuration changes.

$confData = [
    'DeliveryChannel' => [
        'name' => 'default',
        's3BucketName' => $s3Bucket,
        'snsTopicARN' => $snsTopicARN,
    ],
];

$confClient->putDeliveryChannel($confData);

Finally, now that we have our recording configuration and methods for delivery defined, we have to start recording the changes:

$confData = ['ConfigurationRecorderName' => 'default' ];

$confClient->startConfigurationRecorder($confData);

Now that changes to infrastructure within this region of our account are being recorded with notifications sent, we can be notified of and record infrastructure updates.  We can then use additional SNS subscriptions to process the list of items in our infrastructure that have changed, review them for root cause analysis in the event of service issues, use centralized logging along with event correlation to look for system anomalies, and so on.

You can review the processing of Amazon SNS notifications here:

http://blogs.aws.amazon.com/php/post/Tx15276Q7B4NUO0/Receiving-Amazon-SNS-Messages-in-PHP


 

 

Credential Management – Part 1

by Trevor Rowe | on | in Ruby | Permalink | Comments |  Share

When using AWS, it is important to keep your access credentials secure. It can be challenging to make your credentials available to your application securely. The AWS SDK for Ruby provides a number of helpful interfaces for configuring your credentials that help you keep your secrets safe.

This blog post focuses on securely configuring the aws-sdk gem with your credentials. I will follow up with additional blog posts about rotating credentials, and using roles for instances.

Credential Configuration

The aws-sdk gem requires that you provide your access credentials before making a request. The gem tries to locate your credentials in a number of default locations. If it is unable to find your credentials, it raises an error. The locations it searches are:

  • AWS.config
  • ENV
  • EC2 instance metadata
  • Rails Configuration (RAILS_ROOT/config/aws.yml)

AWS.config

You can use AWS.config to statically configure your credentials for all AWS requests. Simply call AWS.config with your :access_key_id and :secret_access_key (you may also provide a :session_token).

require 'aws-sdk'

AWS.config(:access_key_id => '...', :secret_access_key => '...')

Alternatively, you may provide credentials directly to the service interfaces. This can be helpful if you are working with multiple sets of credentials for multiple AWS accounts (or IAM users).

s3 = AWS::S3.new(:access_key_id => '...', :secret_access_key => '...')    

As tempting as it might be, you should never put access credentials in source code. This makes your secrets available to anyone with access to your source code. It also creates a maintenance burden when you need to rotate your credentials. There are many alternatives, including loading credentials from a configuration file that is not tracked with source control.

require 'aws-sdk'
require 'yaml'

# configure aws-sdk gem from a yaml configuration file
AWS.config(YAML.load_file(path_to_configuration_file))

In the preceding example, the YAML file might look like this:

:access_key_id: 'YOUR_ACCESS_KEY_ID'
:secret_access_key: 'YOUR_SECRET_ACCESS_KEY'

ENV

You can alternatively provide credentials to your application via ENV. By default, the aws-sdk gem searches ENV using two different prefixes for your keys (‘AWS’ and ‘AMAZON’).

ENV['AWS_ACCESS_KEY_ID']
ENV['AWS_SECRET_ACCESS_KEY']
ENV['AWS_SESSION_TOKEN']

EC2 Instance Metadata

The aws-sdk gem supports loading credentials from the instance metadata service on Amazon EC2 instances. If you have started your instance using an AWS IAM instance profile, this will just work, no configuration required. I will dive into this deeper in a followup blog post.

Credential Loading in Rails (RAILS_ROOT/config/aws.yml)

If you are using the aws-sdk gem in a Rails application, the gem attempts to load credentials from RAILS_ROOT/config/aws.yml. This file is ERB-parsed and then loaded by YAML. This file should be formatted in the same way as RAILS_ROOT/config/database.yml. It should have one top-level entry for the Rails.env you are running.

# RAILS_ROOT/config/aws.yml
development:
  access_key_id: DEV_ACCESS_KEY_ID
  secret_access_key: DEV_ACCESS_KEY_ID

If this file contains your secrets in plain text and does not use ERB to load them securely, you should not track this file with source control.

You can now read part two in this series: Rotating Credentials.