AWS Developer Tools Blog

Developer Preview: Ruby SDK code generation using Smithy

What is this?

The AWS SDK For Ruby team is happy to announce the developer preview of smithy-ruby, a toolchain that can be used to code generate a “white label” Ruby SDK for your service API using Smithy modeling. An upcoming future version of the AWS SDK For Ruby will use Smithy code generation.

What is Smithy?

Smithy is an interface definition language and set of tools that allows developers to build clients and servers in multiple languages. Smithy models define a service as a collection of resources, operations, and shapes. A Smithy model enables API providers to generate clients and servers in various programming languages, API documentation, test automation, and example code. For more information about Smithy, see the Smithy documentation.

What’s included in the Ruby SDK

Components of a code generated Ruby SDK

A code generated Ruby SDK using Smithy will have generic components and protocol specific components. These components are (in no particular order):

  • Validators (private) – A set of classes that validate Ruby input types against the Smithy model.
  • Builders (private, protocol) – A set of classes that build a protocol specific request using input (i.e. JSON over HTTP).
  • Stubs (private, protocol) – A set of classes that build a protocol specific response using stub data, used for testing.
  • Parsers (private, protocol) – A set of classes that parse a protocol specific response into data structures (i.e. XML over HTTP).
  • Types (public) – A set of classes that represent structure shapes (Plain Old Ruby Objects).
  • Errors (public, protocol) – A set of classes that represent error shapes and protocol specific error classes.
  • Params (private) – A set of modules that convert hash-y input to rigid input types used by the Client operations.
  • Paginators (public) – A set of classes used for traversing paginated operations automatically.
  • Waiters (public) – A set of classes used to wait until an operation reaches a desired state before resuming control back to the client.
  • Client (public) – A class that ties everything together; it is the public interface to the service API. The client is responsible for constructing requests and returning responses using middleware.

For more information about the components, please see the smithy-ruby wiki.

Middleware

Middleware are classes that sit between the client and the server, providing a way to modify the request-response cycle. At minimum, middleware is used to build a request, send a request, and parse a response. Middleware is organized in a stack and are responsible for calling the next middleware.

Middleware stack

In the client, each API operation will have a method that is responsible for creating its own middleware stack and handling the request and response cycle. Seahorse will ship with 6 default middleware. Each middleware will have access to the request, response, and context.

In detail, the middleware components are:

  • Validate – Validates input using the Validator classes if configured to do so. (Optional – client configuration)
  • Build – Builds a protocol specific request (i.e. JSON over HTTP) using the Builder classes and input.
  • HostPrefix – Modifies the endpoint with a host prefix if configured to do so. (Optional – Smithy trait)
  • Send – Sends the request using a protocol specific client (i.e. HTTP client). The middleware may return responses using the Stubs classes if configured to do so.
  • Parse – Parses a protocol specific response (i.e. XML over HTTP) using the Parser classes and the raw service response.
  • Retry – Retries a request for networking errors and any responses with retry-able or throttling errors.

Protocol implementations may also insert their own code generated middleware. Middleware may also be added at runtime to a Client class or Client instance, and to individual operation calls.

Rails JSON Protocol

A Smithy built Ruby SDK needs a protocol implementation to fully function, much like a car needs an engine. As part of this developer preview, we will be including a protocol implementation that we are calling “Rails JSON”. With the Rails JSON protocol definition, a Smithy model can be used to code generate a Ruby SDK that communicates directly with a Rails API over JSON. Neat!

As a demo, the following sections will detail how to setup a Rails service and generate an SDK that can communicate with it.

Setup Rails API Service

Before we can create an SDK, we need a service for it to communicate to. Let’s first create a new Rails API service with: rails new --api sample-service.

Next, echoing rails documentation, let’s create a High Score model with rails generate scaffold HighScore game:string score:integer and run rake db:migrate.

In models/high_score.rb, add a length validation to the game’s name by adding: validates :game, length: { minimum: 2 }. This validation will be used later.

Now it’s time to start our rails app with rails s and verify it’s running on an endpoint such as http://127.0.0.1:3000; we will need this endpoint for later.

If you aren’t able to generate a Rails app, don’t worry, a copy of this sample Rails service lives in smithy-ruby for now.

Add the Smithy model

To generate the SDK, we need the Smithy model that describes the Rails service we just defined. I’ve conveniently defined this in smithy-ruby in high-score-service.smithy. The model tells smithy-ruby to code generate shapes and a client API for the High Score service and to use Rails’ JSON protocol.

Let’s break down some of the important parts.

The first section tells Smithy to create the HighScoreService using the Rails JSON protocol and define its resources and operations. The resource has an identifier (Rails defaults to id), which is used to look up the High Score. The resource has all of the basic Rails CRUD operations: get, create, update, delete, and list

@railsJson
@title("High Score Sample Rails Service")
service HighScoreService {
    version: "2021-02-15",
    resources: [HighScore],
}

/// Rails default scaffold operations
resource HighScore {
    identifiers: { id: String },
    read: GetHighScore,
    create: CreateHighScore,
    update: UpdateHighScore,
    delete: DeleteHighScore,
    list: ListHighScores
}

The next sections define the service shapes. HighScoreAttributes is a shape that returns all of the properties of a High Score. HighScoreParams includes all of the properties that a High Score will need. The @length validation of >2 characters is applied to game.

/// Modeled attributes for a High Score
structure HighScoreAttributes {
    /// The high score id
    id: String,
    /// The game for the high score
    game: String,
    /// The high score for the game
    score: Integer,
    // The time the high score was created at
    createdAt: Timestamp,
    // The time the high score was updated at
    updatedAt: Timestamp
}

/// Permitted params for a High Score
structure HighScoreParams {
    /// The game for the high score
    @length(min: 2)
    game: String,
    /// The high score for the game
    score: Integer
}

Next are the operation shapes. The @http trait is applied to each operation with the expected Rails path.

/// Get a high score
@http(method: "GET", uri: "/high_scores/{id}")
@readonly
operation GetHighScore {
    input: GetHighScoreInput,
    output: GetHighScoreOutput
}

/// Input structure for GetHighScore
structure GetHighScoreInput {
    /// The high score id
    @required
    @httpLabel
    id: String
}

/// Output structure for GetHighScore
structure GetHighScoreOutput {
    /// The high score attributes
    @httpPayload
    highScore: HighScoreAttributes
}

/// Create a new high score
@http(method: "POST", uri: "/high_scores", code: 201)
operation CreateHighScore {
    input: CreateHighScoreInput,
    output: CreateHighScoreOutput,
    errors: [UnprocessableEntityError]
}

/// Input structure for CreateHighScore
structure CreateHighScoreInput {
    /// The high score params
    @required
    highScore: HighScoreParams
}

/// Output structure for CreateHighScore
structure CreateHighScoreOutput {
    /// The high score attributes
    @httpPayload
    highScore: HighScoreAttributes,

    /// The location of the high score
    @httpHeader("Location")
    location: String
}

/// Update a high score
@http(method: "PUT", uri: "/high_scores/{id}")
@idempotent
operation UpdateHighScore {
    input: UpdateHighScoreInput,
    output: UpdateHighScoreOutput,
    errors: [UnprocessableEntityError]
}

/// Input structure for UpdateHighScore
structure UpdateHighScoreInput {
    /// The high score id
    @required
    @httpLabel
    id: String,

    /// The high score params
    highScore: HighScoreParams
}

/// Output structure for UpdateHighScore
structure UpdateHighScoreOutput {
    /// The high score attributes
    @httpPayload
    highScore: HighScoreAttributes
}

/// Delete a high score
@http(method: "DELETE", uri: "/high_scores/{id}")
@idempotent
operation DeleteHighScore {
    input: DeleteHighScoreInput,
    output: DeleteHighScoreOutput
}

/// Input structure for DeleteHighScore
structure DeleteHighScoreInput {
    /// The high score id
    @required
    @httpLabel
    id: String
}

/// Output structure for DeleteHighScore
structure DeleteHighScoreOutput {}

/// List all high scores
@http(method: "GET", uri: "/high_scores")
@readonly
operation ListHighScores {
    output: ListHighScoresOutput
}

/// Output structure for ListHighScores
structure ListHighScoresOutput {
    /// A list of high scores
    @httpPayload
    highScores: HighScores
}

list HighScores {
    member: HighScoreAttributes
}

Generate the SDK

With the model and a Rails service, it’s now time to generate the SDK. Smithy code generation and integration is only available in Java environments. Fortunately, for this demo, the High Score Service SDK has already been generated and committed to the smithy-ruby repo. Download it from here if you are following along!

If you’d like to generate it yourself, or generate your own Smithy model, you can follow the README instructions that detail how to use smithy-ruby in your Gradle project.

Use the SDK

Now we have a Rails service and an SDK. Start up irb with irb -I high_score_service/lib and try it out!

require 'high_score_service'

# Create an instance of HighScoreService's Client.
# This is similar to the AWS SDK Clients.
# Here we use the endpoint of the Rails service.
client = HighScoreService::Client.new(endpoint: 'http://127.0.0.1:3000')

# List all high scores
client.list_high_scores
# => #<struct HighScoreService::Types::ListHighScoresOutput high_scores=[]>

# Try to create a high score
# Should raise an UnprocessableEntityError, let's find out why
 begin
  client.create_high_score(high_score: { score: 9001, game: 'X' })
rescue => e
  puts e.data
  # => #<struct HighScoreService::Types::UnprocessableEntityError errors={"game"=>["is too short (minimum is 2 characters)"]}>
end

# Actually create a high score
client.create_high_score(high_score: { score: 9001, game: 'Frogger' })
# => #<struct HighScoreService::Types::CreateHighScoreOutput

# List high scores again
resp = client.get_high_score(id: '1')
resp.high_score
# => #<struct HighScoreService::Types::HighScoreAttributes id=1, game="Frogger", score=9001 ... >

As an exercise, try out the delete_high_score and update_high_score operations.

Future Plans

Looking forward, smithy-ruby will be used to generate the new service client versions (gem version 2, core version 4) of AWS SDK For Ruby.

We’d like to explore more Smithy Ruby and Rails API use cases. Perhaps a Smithy model can be parsed and translated into a set of rails new and rails generate commands; going further, perhaps a “server-side SDK” can be a pluggable Rails engine that handles building and parsing of concrete types and protocols.

Feedback

If you have any questions, comments, concerns, ideas, or other feedback, please create an issue or discussion in the smithy-ruby repository. We welcome any SDK design feedback and improvements, and we especially welcome any community contributions.

Thanks for reading!

-Matt