Category: Ruby


Introducing Support for Generating Ruby SDKs in Amazon API Gateway

by Jingyi Chen | on | in Ruby | Permalink | Comments |  Share

We’re excited to announce support for generating Ruby SDKs from Amazon API Gateway. The Ruby SDKs you generated are compatible with Ruby 1.9 and later. Generated SDKs have first-class support for API keys, custom or AWS Identity and Access Management (IAM) authentication, automatic and configurable retries, exception handling, and all privileges of aws-sdk-core version 3 has as well. In this blog post, we’ll walk through how to create an example API and generate a Ruby SDK from that API. We also explore various features of the generated SDK. In this post, we assume you have some familiarity with API Gateway concepts.

Creating an example API

To start, let’s create an sample API by using the API Gateway console.
Open the API Gateway console, choose Create API, and then choose Example API. Then choose Import to create the example API.

This simple, example API has four straightforward operations:

  • A GET on the API root resource that returns HTML describing the API
  • A GET on the /pets resource that returns a list of Pets
  • A POST on the /pets resource that creates a new Pet
  • A GET on the /pets/{petId} resource that returns a specific Pet by ID

You can find more information about this example in the API Gateway documentation.

Deploying the API

Next, let’s deploy our API to a stage.
From Actions choose Deploy API.

On the stage deployment page, name the stage Test, and then choose Deploy.

After deploying, the SDK Generation tab is available. For Platform, choose Ruby
For Service Name, type Pet.

Choose Generate SDK, and then extract the downloaded SDK package.

The following are the configuration options available for the Ruby platform:

  • Service Name – Used to generate the Ruby gem namespace for your APIs.
  • Ruby Gem Name – The name of the Ruby gem your generated SDK code will be placed under. If you don’t provide a name, this defaults to the service name in lowercase, with the “sdk” suffix.
  • Ruby Gem Version – The version number for the generated Ruby gem. If you don’t provide a version number, this defaults to 1.0.0 if not provided.

These are basic Ruby gem configuration options. You can customize your Ruby gemspec in the generated SDK gem later.

Using the generated Ruby SDK gem

Navigate to the location of your downloaded SDK gem. The directory structure looks like the following.

Note
/features and /spec directories are currently left empty for integration and unit tests that you can add to the SDK. The generated SDK is fully documented for operations and shapes in the source code.

Exploring the SDK

Let’s explore the SDK by building a Ruby gem from the generated source, as follows.

# change to /pet-sdk directory
cd pet-sdk

# build the generated gem
gem build pet-sdk.gemspec
# then you can see pet-sdk-1.0.0.gem is available

Then, install the gem, as follows.

gem install pet-sdk-1.0.0.gem

Finally, create the client.

require 'pet-sdk'

client = Pet::Client.new

Features in the client

Now you have your own client that includes multiple features from the official AWS SDK for Ruby. These include default exponential backoff retries, HTTP wire logging options, configurable timeouts, and more.

For example:

require 'pet-sdk'
client = Pet::Client.new(
  http_wire_trace: true,
  retry_limit: 5,
  http_read_timeout: 50
 )

Making API calls

Let’s see all the API methods that are available and use your SDK’s built-in parameter validators to make a successful API call:

client.operation_names
# => [:create_pet, :get_api_root, :get_pet, :get_pets]

# get me all my pets
resp = client.get_pets

You should see a response like the following.

# I want the cat
client.get_pet
# ArgumentError: missing required parameter params[:pet_id]

# providing :pet_id
client.get_pet(pet_id: 2)
# ArgumentError: expected params[:pet_id] to be a String, got value 2 (class: Fixnum) instead.

# fix the value type
resp = client.get_pet(pet_id: "2")

Now you can see a correct response like the following.
If you have some familiarity with the AWS SDK for Ruby, you should find the experience similar to using an AWS service client.

Generate a Ruby SDK from an API

In addition to using the API Gateway console to generate a Ruby SDK, the get_sdk API is available in all of the AWS SDKs and tools, including the AWS SDK for Ruby.

For this example, we assume that you have some familiarity with the AWS SDK for Ruby. You can find a quick introduction to the SDK for Ruby here.

require 'aws-sdk-apigateway'

client = Aws::ApiGateway::Client.new(region: 'us-west-2')
resp = client.get_sdk({
  rest_api_id: MY_REST_API_ID, # required
  stage_name: DEPLOY_STAGE_NAME, # required
  sdk_type: "ruby", # required
  parameters: {
    "service.name" => "PetStore", # required
    "ruby.gem-name" => "pet",
    "ruby.gem-version" => "0.0.1"
  },
})

Final thoughts

This post highlights how to generate a Ruby client SDK for an API in API Gateway, and how to call the API using the generated SDK in an application. For more information about using a generated SDK, see your README.md file in the uncompressed generated SDK gem folder. Details of example usage of your API are also generated in source file documentation blocks.

Feedback

Please share your questions, comments, and issues with us on GitHub. Feel free to open a support ticket with AWS Support if you find an issue with your API model. You can also catch us in our Gitter channel.

Announcing the Modularized AWS SDK for Ruby (Version 3)

by Alex Wood | on | in Ruby | Permalink | Comments |  Share

We’re excited to announce today’s stable release of version 3 of the AWS SDK for Ruby. The SDK is now available with over 100 service-specific gems (starting with aws-sdk-*, such as aws-sdk-s3) on RubyGems. You can find a full list of available service gems can be found at our GitHub landing page.

Features

Version 3 of the AWS SDK for Ruby modularizes the monolithic SDK into service-specific gems, for example, aws-sdk-s3 and aws-sdk-dynamodb. Now each service gem uses strict semantic versioning, along with the benefits of continuous delivery of AWS API updates. With modularization, you can pick and choose which service gems your application or library requires, and update service gems independently of each other.

These new service-specific gems use statically generated code, rather than runtime-generated clients and types. This provides human-readable stack traces and code for API clients. Additionally, version 3 eliminates many thread safety issues by removing Ruby autoload statements. When you require a service gem, such as aws-sdk-ec2, all of the code is loaded upfront, avoiding sync issues with autoload.

Furthermore, the SDK provides AWS Signature Version 4 signing functionality as a separate gem aws-sigv4. This gem provides flexible signing usage for both AWS requests and customized scenarios.

Upgrading

We’ve provided a detailed upgrading guide with this release, which covers different upgrade scenarios. In short, the public-facing APIs are compatible, and so changes you need to make are focused on your Gemfile and require statements.

Most users of the SDK have a setup similar to this:

# Gemfile
gem 'aws-sdk', '~> 2'
# Code Files
require 'aws-sdk'

s3 = Aws::S3::Client.new

ddb = Aws::DynamoDB::Client.new

# etc.

If that’s you, the quickest migration path is to simply change your Gemfile like so:

# Gemfile
gem 'aws-sdk', '~> 3'

However, this will pull in many new dependencies, as each service client has its own individual gem. As a direct user, you can also change to using only the service gems actually required by your project, which is the recommended path. This would involve a change to both your Gemfile and to the code files where your require statements live, like so:

# Gemfile
gem 'aws-sdk-s3', '~> 1'
gem 'aws-sdk-dynamodb', '~> 1'
# Code Files
require 'aws-sdk-s3'
require 'aws-sdk-dynamodb'

s3 = Aws::S3::Client.new
ddb = Aws::DynamoDB::Client.new

# etc.

Other upgrade cases are covered in the guide.

Feedback

Please share your questions, comments, issues, etc. with us on GitHub. You can also catch us in our Gitter channel.

Upgrading from Version 2 to Version 3 of the AWS SDK for Ruby

by Trevor Rowe | on | in Ruby | Permalink | Comments |  Share

Recently we announced the modularization of the AWS SDK for Ruby. This blog post will focus on how to upgrade your application to use the new service specific gems.

This blog post is divided up into sections based on how you currently depend on the AWS SDK for Ruby today. Find the section below that describes how you load the SDK today, and it will guide you in upgrading. Since version 3 is backwards compatible with version 2, you should not need to make additional changes beyond those described below.

Bunder: gem ‘aws-sdk’, ‘~>2’

Congratulations! You are following recommended best practices for how to depend on the SDK today. The simplest path to upgrade is to change the version from 2 to 3.

#gem 'aws-sdk', '~> 2'
gem 'aws-sdk', '~> 3'

See the section about using service specific gems below.

Bundler: gem ‘aws-sdk’ (without version)

It is not recommended to depend on the SDK without a major version constraint. Fortunately, version 3 is backwards compatible with version 2. Bundle updating your dependencies will work, but consider yourself lucky! You should add the version constraint to protect from future major version changes:

#gem 'aws-sdk'
gem 'aws-sdk', '~> 3'

See the section about using service specific gems below.

Bundler: gem ‘aws-sdk-core’, ‘~> 2’

The aws-sdk-core gem changes signification from version 2 to 3. In version 3, the core gem no longer defines any services. It will only contain shared utilities, such as credential providers, serializers, etc. To upgrade you must make two changes:

  • Change your bundler dependency
  • Change your ruby require statements

In bundler, replace aws-sdk-core, with aws-sdk:

#gem 'aws-sdk-core', '~> 2'
gem 'aws-sdk', '~> 3'

In code, replace any require statements on aws-sdk-core with aws-sdk.

#require 'aws-sdk-core'
require 'aws-sdk'

See the section about using service specific gems below.

Bundler: gem ‘aws-sdk-core’ (without version)

If you happen to bundle update before changing your Gemfile, your application will be broken. Version 3 of the aws-sdk-core gem no longer defines service modules and clients. It is a shared dependency of the 75+ service gems. To upgrade you must make two changes:

  • Change your bundler dependency
  • Change your ruby require statements

In bundler, replace aws-sdk-core, with aws-sdk:

#gem 'aws-sdk-core'
gem 'aws-sdk', '~> 3'

In code, replace any require statements on aws-sdk-core with aws-sdk.

#require 'aws-sdk-core'
require 'aws-sdk'

See the section about using service specific gems below.

Bundler: gem ‘aws-sdk-resource’ (with or without version)

In version 3, the aws-sdk-resources gem has been removed. This gem will not receive any further updates. Each service gem contains both the client interface, and the resource interfaces. To upgrade you must make two changes:

  • Change your bundler dependency
  • Change your ruby require statements
#gem 'aws-sdk-resources', '~> 2'
gem 'aws-sdk', '~> 3'

In code, replace any require statements on aws-sdk-resources with aws-sdk.

#require 'aws-sdk-core'
require 'aws-sdk'

See the section about using service specific gems below.

Using the Service Specific Gems

Each of the instructions above suggested using version 3 of the aws-sdk gem. This will work and is the shortest path to upgrading. It will however install 75+ service specific gems. You may choose to replace your dependency on the aws-sdk gem with service specific gems.

If my application depends on Amazon DynamoDB and Amazon S3, I could make the following changes:

In my Gemfile:

#gem 'aws-sdk', '~> 3'
gem 'aws-sdk-dynamodb', '~> 1'
gem 'aws-sdk-s3', '~> 1'

In my code:

#require 'aws-sdk'
require 'aws-sdk-s3'
require 'aws-sdk-dynamodb'

If you are a library maintainer, and you depend on the AWS SDK for Ruby, you should use service specific gems. Do no force your users to install every AWS service gem if you only depend on one.

Conclusion

Upgrading should be very simple. If you encounter any backwards incompatible changes, please open a GitHub issue. The modularized SDK will be in preview for a short period to hopefully catch these issues before going GA. You can also catch us in the gitter channel.

AWS SDK for Ruby Modularization (Version 3)

by Trevor Rowe | on | in Ruby | Permalink | Comments |  Share

Version 3 of the AWS SDK for Ruby is available now as a preview release. This version modularizes the monolithic SDK into service specific gems. Aside from gem packaging differences, version 3 interfaces are backwards compatible with version 2.

You can install individual gems like so:

$ gem install aws-sdk-s3 --version 1.0.0.rc1

You can install everything using the aws-sdk gem:

$ gem install aws-sdk --version 3.0.0.rc1

To see a complete list of gems, checkout the project README for a list of supported services and gems.

Motivation

Modularization allows us to make some long requested changes to the SDK. Some of these changes were not reasonable when we shipped a single monolithic gem with 75+ services. Some of the primary motivating factors for breaking up the monolith include:

  • To provide better versioning information. When 75+ services share a single gem, it is difficult to communicate when a change is meaningful to a user. We can now use strict semantic versioning for each gem.
  • To improve our ability to deliver AWS API updates continuously. The number of new services and updates has been significantly increasing the frequency with which we update. We want to avoid situations where one update is blocked by an unrelated service. We can now continuously deliver updates per service gem.
  • Remove the usage of Ruby `autoload` statements. When you require a service gem, such as aws-sdk-ec2, all of the code is loaded and ready to use. This should eliminate a large number of thread safety issues that users encounter due to the use of autoload.
  • A large amount of the dynamic runtime has been replaced with code generation. This allows users to reason better about what code is doing, receive better stack traces, improve performance, etc.

What Has Changed?

Our intent for the modularization is to keep SDK interfaces backwards compatible. You may need to modify your gem dependency on the AWS SDK for Ruby. The aws-sdk and aws-sdk-core gems have been bumped to version 3.0 to protect users from package level changes.

* Every service has a gem, such as aws-sdk-s3.
* The aws-sdk-core gem now only contains shared utilities.
* The aws-sdk-resources is obsolete. Service gems contain both client and resource interfaces.
* The aws-sdk gem now has a dependency on 75+ service gems.

Here is a diagram showing the dependencies of the aws-sdk gem across its major versions.

gem-diagram

Why Bump to Version 3?

The version 2 aws-sdk-core gem includes code that defines 75+ service modules and shared utilities. It is important to prevent a service specific gem, such as aws-sdk-s3 and the core gem from both defining the same interfaces.

While we have worked hard to ensure full backwards compatibility in the service interfaces, a small number of private internal interfaces have been removed or changed. For users that have relied on these un-documented interfaces, this will prevent unexpected breaks with a gem update. Some of these changes include:

  • Removed the internal runtime methods Aws.add_service and Aws.service_added. These methods were used by the runtime to detect when a service was autoloaded.
  • Removed the internal Aws::Signers module and the various signature classes therein. These classes were marked with @api private. They are now available as separate gems:
    • aws-sigv4
    • aws-sigv2

Migrating Code From Version 2 to Version 3

Migrating should be very simple. If you depend on aws-sdk, then you do not need to change anything. If you depend on aws-sdk-resources or aws-sdk-core, replace these with a dependency on one of the following:

* aws-sdk ~> 3.0
* Service specific gems, such as aws-sdk-s3 ~> 1.0

You will also need to replace your require statements. You should no longer call require "aws-sdk-resources" or require "aws-sdk-core". A follow-up blog post provides detailed instructions on upgrading.

Questions?

Join us in our Gitter channel with your questions and feedback. The modularized released is currently published as a preview gem (rc1). We would love for you to try things out and to share feedback for these are GA.

General Availability Release of the aws-record Gem

by Alex Wood | on | in Ruby | Permalink | Comments |  Share

Today, we’re pleased to announce the GA release of version 1.0.0 of the aws-record gem.

What Is aws-record?

In version 1 of the AWS SDK for Ruby, the AWS::Record class provided a data mapping abstraction over Amazon DynamoDB operations. Earlier this year, we released the aws-record developer preview as a separately packaged library to provide a similar data mapping abstraction for DynamoDB, built on top of the AWS SDK for Ruby version 2. After customer feedback and some more development work, we’re pleased to move the library out of developer preview to general availability.

How to Include the aws-record Gem in Your Project

The aws-record gem is available now from RubyGems:

 

gem install aws-record

 

You can also include it in your project’s Gemfile:

 

# Gemfile
gem 'aws-record', '~> 1.0'

 

This automatically includes a dependency on the aws-sdk-resources gem, major version 2. Be sure to include the aws-sdk or aws-sdk-resources gem in your Gemfile if you need to lock to a specific version, like so:

 

 # Gemfile
gem 'aws-record', '~> 1.0'
gem 'aws-sdk-resources', '~> 2.5'

 

Working with DynamoDB Tables Using the aws-record Gem

Defining an Aws::Record Model

The aws-record gem provides the Aws::Record module, which you can include in a class definition. This decorates your class with a variety of helper methods that can simplify interactions with Amazon DynamoDB. For example, the following model uses a variety of preset attribute definition helper methods and attribute options:

 

require 'aws-record'

class Forum
  include Aws::Record  

  string_attr     :forum_uuid, hash_key: true
  integer_attr    :post_id,    range_key: true
  string_attr     :author_username
  string_attr     :post_title
  string_attr     :post_body
  string_set_attr :tags,       default_value: Set.new 
  datetime_attr   :created_at, database_attribute_name: "PostCreatedAtTime"
  boolean_attr    :moderation, default_value: false
end

 

Using Validation Libraries with an Aws::Record Model

The aws-record gem does not come with a built-in validation process. Rather, it is designed to be a persistence layer, and to allow you to bring your own validation library. For example, the following model includes the popular ActiveModel::Validations module, and has defined a set of validations that will be run when we attempt to save an item:

 

require 'aws-record'
require 'active_model'

class Forum
  include Aws::Record
  include ActiveModel::Validations

  string_attr     :forum_uuid, hash_key: true
  integer_attr    :post_id,    range_key: true
  string_attr     :author_username
  string_attr     :post_title
  string_attr     :post_body
  string_set_attr :tags,       default_value: Set.new 
  datetime_attr   :created_at, database_attribute_name: "PostCreatedAtTime"
  boolean_attr    :moderation, default_value: false 


  validates_presence_of :forum_uuid, :post_id, :author_username
  validates_length_of :post_title, within: 4..30
  validates_length_of :post_body,  within: 2..5000
end

 

Creating a DynamoDB Table for a Model with Aws::Record::TableMigration

The aws-record gem provides a helper class for table operations, such as migrations. If we wanted to create a table for our Forum model in DynamoDB, we would run the following migration:

 

migration = Aws::Record::TableMigration.new(Forum)
migration.create!(
  provisioned_throughput: {
    read_capacity_units: 5,
    write_capacity_units: 2
  }
)
migration.wait_until_available

 

You can write these migrations in your Rakefile or as standalone helper scripts for your application. Because you don’t need to update your table definition for additions of non-key attributes, you may find that you’re not running migrations as often for your Aws::Record models.

Working with DynamoDB Items Using the aws-record Gem

Creating and Persisting a New Item

Using the example model above, once it has been created in the DynamoDB remote end using Aws::Record::TableMigration (or if it already existed in the remote end), it is simple to create and save a new item:

 

post = Forum.new(
  forum_uuid: FORUM_UUID,
  post_id: 1,
  author_username: "Author One",
  post_title: "Hello!",
  post_body: "Hello, world!"
)
post.created_at = Time.now
post.save # Performs a put_item call.

 

You can set attributes when you initialize a new item and with setter methods that are defined for you automatically.

Finding and Modifying an Item

A class-level method #find is provided to look up items from DynamoDB using your model’s key attributes. After setting a few new attribute values, calling #save will make an update call to DynamoDB, reflecting only the item changes you’ve made. This is important for users who are fetching items with projections (which may not include all attributes), or using single-table inheritance patterns (who may not have modeled all attributes present in a remote item), to avoid clobbering unmodeled or non-included attribute values.

 

post = Forum.find(forum_uuid: FORUM_UUID, post_id: 1)
post.post_title = "(Removed)"
post.post_body = "(Removed)"
post.moderation = true
post.save # Performs an update_item call on dirty attributes only.

 

There is also a class-level method to directly build and make an update call to DynamoDB, using key attributes to identify the item and non-key attributes to form the update expression:

 

Forum.update(
  forum_uuid: FORUM_UUID,
  post_id: 1,
  post_title: "(Removed)",
  post_body: "(Removed)",
  moderation: true
)

 

The preceding two code examples are functionally equivalent. You’ll have the same database state after running either snippet.

A Note on Dirty Tracking

In our last example, we talked about how item updates only reflect changes to modified attributes. Users of ActiveRecord or similar libraries will be familiar with the concept of tracking dirty attribute values, but aws-record is a bit different. That is because DynamoDB supports collection attribute types, and in Ruby, collection types are often modified through object mutation. To properly track changes to an item when objects can be changed through mutable state, Aws::Record items will, by default, keep deep copies of your attribute values when loading from DynamoDB. Attribute changes through mutation, like this example, will work the way you expect:

 

post = Forum.find(forum_uuid: FORUM_UUID, post_id: 1)
post.tags.add("First")
post.dirty? # => true
post.save # Will call update_item with the new tags collection.

 

Tracking deep copies of attribute values has implications for performance and memory. You can turn off mutation tracking at the model level. If you do so, dirty tracking will still work for new object references, but will not work for mutated objects:

 

class NoMTModel
  include Aws::Record
  disable_mutation_tracking
  string_attr :key, hash_key: true
  string_attr :body
  map_attr    :map
end

item = NoMTModel.new(key: "key", body: "body", map: {})
item.save # Will call put_item
item.map[:key] = "value"
item.dirty? # => false, because we won't track mutations to objects
item.body = "New Body"
item.dirty? # => true, because we will still notice reassignment
# Will call update_item, but only update :body unless we mark map as dirty explicitly.
item.save

 

Try the aws-record Gem Today!

We’re excited to hear about what you’re building with aws-record. Feel free to leave your feedback in the comments, or open an issue in our GitHub repo. Read through the documentation and get started!

Introducing the Aws::Record Developer Preview

by Alex Wood | on | in Ruby | Permalink | Comments |  Share

We are happy to announce that the aws-record gem is now in Developer Preview and available for you to try.

What Is Aws::Record?

In version 1 of the AWS SDK for Ruby, the AWS::Record class provided a data mapping abstraction over Amazon DynamoDB operations. As version 2 of the AWS SDK for Ruby was being developed, many of you asked for an updated version of the library.

The aws-record gem provides a data mapping abstraction for DynamoDB built on top of the AWS SDK for Ruby version 2.

Using Aws::Record

You can download the aws-record gem from RubyGems by including the --pre flag in a gem installation:

gem install 'aws-record' --pre

You can also include it in your Gemfile. Do not include a version lock yet, so that bundler can find the pre-release version:

# Gemfile
gem 'aws-record'

Defining a Model

To create an aws-record model, include the Aws::Record module in your class definition:

require 'aws-record'

class Forum
  include Aws::Record
end

This will decorate your class with helper methods you can use to create a model compatible with DynamoDB’s table schemas. You might define keys for your table:

require 'aws-record'

class Forum
  include Aws::Record
  string_attr  :forum_uuid, hash_key: true
  integer_attr :post_id,    range_key: true
end

When you use these helper methods, you do not need to worry about how to define these attributes and types in DynamoDB. The helper methods and marshaler classes are able to define your table and item operations for you. The aws-record gem comes with predefined attribute types that cover a variety of potential use cases:

require 'aws-record'

class Forum
  include Aws::Record
  string_attr   :forum_uuid, hash_key: true
  integer_attr  :post_id,    range_key: true
  string_attr   :author_username
  string_attr   :post_title
  string_attr   :post_body
  datetime_attr :created_at
  map_attr      :post_metadata
end

Creating a DynamoDB Table

The aws-record gem provides a helper class for table operations, such as migrations. If we wanted to create a table for our Forum model in DynamoDB, we would run the following migration:

require 'forum' # Depending on where you defined the class above.

migration = Aws::Record::TableMigration.new(Forum)

migration.create!(
  provisioned_throughput: {
    read_capacity_units: 10,
    write_capacity_units: 4
  }
)

migration.wait_until_available # Blocks until table creation is complete.

Operations with DynamoDB Items

With a model and table defined, we can perform operations that relate to items in our table. Let’s create a post:

require 'forum'
require 'securerandom'

uuid = SecureRandom.uuid

post = Forum.new
post.forum_uuid = uuid
post.post_id = 1
post.author_username = "User1"
post.post_title = "Hello!"
post.post_body = "Hello Aws::Record"
post.created_at = Time.now
post.post_metadata = {
  this_is_a: "Post",
  types_supported_include: ["String", "Integer", "DateTime"],
  how_many_times_ive_done_this: 1
}

post.save # Writes to the database.

This example shows us some of the types that are supported and serialized for you. Using the key we’ve defined, we can also find this object in our table:

my_post = Forum.find(forum_uuid: uuid, post_id: 1)
my_post.post_title # => "Hello!"
my_post.created_at # => #<DateTime: 2016-02-09T14:39:07-08:00 ((2457428j,81547s,0n),-28800s,2299161j)>

You can use the same approach to save changes or, as shown here, you can delete the item from the table:

my_post.delete! # => true

At this point, we know how to use Aws::Record to perform key-value store operations powered by DynamoDB and have an introduction to the types available for use in our tables.

Querying, Scanning, and Collections

Because it is likely that you’re probably doing Query and Scan operations in addition to key-value operations, aws-record provides support for integrating them with your model class.

When you include the Aws::Record module, your model class is decorated with #query and #scan methods, which correspond to the AWS SDK for Ruby client operations. The response is wrapped in a collection enumerable for you. Consider the following basic scan operation:

Forum.scan # => #<Aws::Record::ItemCollection:0x007ffc293ec790 @search_method=:scan, @search_params={:table_name=>"Forum"}, @model=Forum, @client=#<Aws::DynamoDB::Client>>

No client call has been made yet: ItemCollection instances are lazy, and only make client calls only when needed. Because they provide an enumerable interface, you can use any of Ruby’s enumerable methods on your collection, and your result page is saved:

resp = Forum.scan
resp.take(1) # Makes a call to the underlying client. Returns a 'Forum' object.
resp.take(1) # Same result, but does not repeat the client call.

Because the Aws::Record::ItemCollection uses version 2 ofthe AWS SDK for Ruby, pagination support is built-in. So, if your operation requires multiple DynamoDB client calls due to response truncation, ItemCollection will handle the calls required in your enumeration:

def author_posts
  Forum.scan.inject({}) do |acc, post|
    author = post.author_username
    if acc[author]
      acc[author] += 1
    else
      acc[author] = 1
    end
    acc
  end
end

The same applies for queries. Your query result will also be provided as an enumerable ItemCollection:

def posts_by_forum(uuid)
  Forum.query(
    key_condition_expression: "#A = :a",
    expression_attribute_names: {
      "#A" => "forum_uuid"
    },
    expression_attribute_values: {
      ":a" => uuid
    }
  )
end

Given this functionality, you have the flexibility to mix and match Ruby’s enumerable functionality with DynamoDB filter expressions, for example, to curate your results. These two functions return the same set of responses:

def posts_by_author_in_forum(uuid, author)
  posts_by_forum(uuid).select do |post|
    post.author_username == author
  end
end

def posts_by_author_in_forum_with_filter(uuid, author)
  Forum.query(
    key_condition_expression: "#A = :a",
    filter_expression: "#B = :b",
    expression_attribute_names: {
      "#A" => "forum_uuid",
      "#B" => "author_username"
    },
    expression_attribute_values: {
      ":a" => uuid,
      ":b" => author
    }
  )
end

Support for Secondary Indexes

Aws::Record also supports both local and global secondary indexes. Consider this modified version of our Forum table:

require 'aws-record'

class IndexedForum
  include Aws::Record

  string_attr   :forum_uuid, hash_key: true
  integer_attr  :post_id,    range_key: true
  string_attr   :author_username
  string_attr   :post_title
  string_attr   :post_body
  datetime_attr :created_at
  map_attr      :post_metadata

  global_secondary_index(:author,
    hash_key: :author_username,
    projection: {
      projection_type: "INCLUDE",
      non_key_attributes: ["post_title"]
    }
  )

  local_secondary_index(:by_date,
    range_key: :created_at,
    projection: {
      projection_type: "ALL"
    }
  )
end

You can see the table’s attributes are the same, but we’ve included a couple potentially useful indexes.

  • :author: This uses the author name as a partition, which provides a way to search across forums by author user name without having to scan and filter. Take note of the projection, because your global secondary index results will only return the :forum_uuid, :post_id, :author_username, and :post_title. Other attributes will be missing from this projection, and you would have to hydrate your item by calling #reload! on the item instance.
  • :by_date: This provides a way to sort and search within a forum by post creation date.

To create this table with secondary indexes, you create a migration like we did before:

require 'indexed_forum'

migration = Aws::Record::TableMigration.new(IndexedForum)

migration.create!(
  provisioned_throughput: {
    read_capacity_units: 10,
    write_capacity_units: 4
  },
  global_secondary_index_throughput: {
    author: {
      read_capacity_units: 5,
      write_capacity_units: 3
    }
  }
)

migration.wait_until_available

You can use either of these indexes with the query interface:

require 'indexed_forum'

def search_by_author(author)
  IndexedForum.query(
    index_name: "author",
    key_condition_expression: "#A = :a",
    expression_attribute_names: {
      "#A" => "author_username"
    },
    expression_attribute_values: {
      ":a" => author
    }
  )
)

Secondary indexes can be a powerful performance tool, and aws-record can simplify the process of managing them.

Get Involved!

Please download the gem, give it a try, and let us know what you think. This project is a work in progress, so we welcome feature requests, bug reports, and information about the kinds of problems you’d like to solve by using this gem. And, as with other SDKs and tools we produce, we’d also be happy to look at contributions.

You can find the project on GitHub at https://github.com/awslabs/aws-sdk-ruby-record

Please reach out and let us know what you think!

AWS SDK for Ruby Office Hour

by Trevor Rowe | on | in Ruby | Permalink | Comments |  Share

The AWS SDKs and Tools team invites you to the first-ever online office hour hosted by the maintainers of the AWS SDK for Ruby. It will be held via Google Hangouts at 11:00am-12:00pm PDT (UTC -7:00) on Tuesday 6/30. If you don’t have one already, you will need to create an account with Google to join the video chat.

This first office hour will be driven by customer questions. We expect to focus on questions about the SDK, but any questions related to Ruby development on AWS are welcome. We’re excited to meet you and help you be successful in developing Ruby applications on AWS!

Please register for the event, add it to your calendar, and join the office hour next Monday.

RailsConf 2015 Recap

by Alex Wood | on | in Ruby | Permalink | Comments |  Share

Recently, Trevor, Loren, and myself from the AWS SDK team attended RailsConf in Atlanta. We had a great time at the conference and enjoyed connecting with many of you there.

Our Rails on Amazon Web Services Workshop

At RailsConf, we ran a workshop called Deploy and Manage Ruby on Rails Apps on AWS. It was an amazing experience for us, with attendees of all experience levels gettings hands-on experience not only deploying to AWS, but learning about the tools we’ve made to help make integrations easier.

For those of you who could not make it, you can still give this workshop a try!

  • Detailed step-by-step instructions, the same as we provided to attendees, are available here.
  • You can also follow along with the presentation recording on YouTube.
  • Code for the sample app is available on GitHub.
  • If you’d like to try using Amazon Relational Database Service instead of using AWS OpsWorks managed MySQL, you can reference our blog post on that topic as well.

Continuing the Conversation

We hope to see more of you on the conference trail again soon! Apropos of this, it is worth mentioning that AWS re:Invent registration is open at the time of writing. We will be there, and we hope to see you there!

Uploading Files to Amazon S3

by Trevor Rowe | on | in Ruby | Permalink | Comments |  Share

I blogged previously about downloading objects from Amazon S3 using the version 2 AWS SDK for Ruby. It was requested that I write about uploading objects as well.

Managed File Uploads

The simplest and most common task is upload a file from disk to a bucket in Amazon S3. This is very straightforward when using the resource interface for Amazon S3:

s3 = Aws::S3::Resource.new

s3.bucket('bucket-name').object('key').upload_file('/source/file/path')

You can pass additional options to the Resource constructor and to #upload_file. This expanded example demonstrates configuring the resource client, uploading a public object and then generating a URL that can be used to download the object from a browser.

s3 = Aws::S3::Resource.new(
  credentials: Aws::Credentials.new('akid', 'secret'),
  region: 'us-west-1'
)

obj = s3.bucket('bucket-name').object('key')
obj.upload_file('/source/file/path', acl:'public-read')
obj.public_url
#=> "https://bucket-name.s3-us-west-1.amazonaws.com/key"

This is the recommended method of using the SDK to upload files to a bucket. Using this approach has the following benefits:

  • Manages multipart uploads for objects larger than 15MB.
  • Correctly opens files in binary mode to avoid encoding issues.
  • Uses multiple threads for uploading parts of large objects in parallel.

Other Methods

In addition to Aws::S3::Object#upload_file, you can upload an object using #put or using the multipart upload APIs.

PUT Object

For smaller objects, you may choose to use #put instead. The #put method accepts an optional body, which can be a string or any IO object.

obj = s3.bucket('bucket-name').object('key')

# from a string
obj.put(body:'Hello World!')

# from an IO object
File.open('/source/file', 'rb') do |file|
  obj.put(body:file)
end

Multipart APIs

I recommend you use #upload_file whenever possible. If you need to manage large object copies, then you will need to use the multipart interfaces. There are restrictions on the minimum file, and part sizes you should be aware of. Typically these are reserved for advanced use cases.

Feedback

I’d love to hear feedback. If you find the AWS SDK for Ruby lacks a utility for working with Amazon S3, I’d love to hear about it. Please feel free to open a GitHub issue or drop into our Gitter channel.

Verifying Amazon SNS Message Authenticity

by Trevor Rowe | on | in Ruby | Permalink | Comments |  Share

You can now use version 2 of the AWS SDK for Ruby to verify the signatures of Amazon SNS messages. To help prevent spoofing attacks, you should verify messages are sent by Amazon SNS.

The new verifier follows the documented best practices for verification, including:

  • Always use HTTPS when getting the certificate from Amazon SNS.
  • Validate the authenticity of the certificate.
  • Verify the certificate was received from Amazon SNS.

Basic Usage

Usage is straightforward. Construct a message verifier and call one of two methods. The given message body should be the JSON document string of the message.

verifier = Aws::SNS::MessageVerifier.new

verifier.authentic?(message_body)
#=> returns true or false

verifier.authenticate!(message_body)
#=> returns true or raises a VerificationError

You can use one instance of Aws::SNS::MessageVerifier.new to verify multiple messages.

Feedback

As always, we love to hear your feedback. It helps us prioritize our development efforts. In fact, this feature was added by customer request. Feel free to join our Gitter channel or open a GitHub issue.