Introducing the Smithy CLI

The Smithy team is excited to announce the official release of the Smithy CLI. Smithy is an open-source Interface Definition Language (IDL) for web services created by AWS. AWS uses Smithy to model services, generate server scaffolding and rich clients in multiple languages, and generate the AWS SDKs. Smithy enables large-scale collaboration on APIs through its extensible meta-model and pluggable design. It is purpose-built for enabling code generation in multiple languages, can be extended with custom traits, enables automatic API standards enforcement, and is protocol-agnostic. Smithy’s design is rooted in our experience building thousands of service APIs and developing complex SDKs within Amazon. To learn more, check out smithy.io, and please watch the introductory talk from Michael Dowling, Smithy’s Principal Engineer.

Currently, most developers build their Smithy models using Gradle and the smithy-gradle plugin. However, a developer would need to have an installation of Java, and knowledge of how Gradle tooling works just to build their models. This is overly complex for developers who aren’t already familiar with these tools, and detracts from the intended experience of modeling with Smithy.

With the Smithy CLI, developers can build their models quicker and with a single command, without any knowledge of Java or Gradle. The Smithy CLI is now available to download on MacOS, Linux, and Windows platforms.

Getting Started

The Smithy CLI enables you to quickly iterate on your Smithy models. With this tool, you can easily build your models, run ad-hoc validation on your models, compare models for differences, and query them. To install the Smithy CLI on MacOS with Homebrew, you can run the following commands:

$ brew tap smithy-lang/tap
$ brew install smithy-cli

For install instructions on other platforms or for detailed instructions on installation, you can view the installation guide. Now that you have the Smithy CLI installed, you can view the ‘help’ information by using the --help flag:

$ smithy --help
Usage: smithy [-h | --help] [--version] <command> [<args>]
 
Available commands:
    validate    Validates Smithy models.
    build       Builds Smithy models and creates plugin artifacts for each projection found in smithy-build.json.
    diff        Compares two Smithy models and reports differences.
    ast         Reads Smithy models in and writes out a single JSON AST model.
    select      Queries a model using a selector.
    clean       Removes Smithy build artifacts.
    migrate     Migrate Smithy IDL models from 1.0 to 2.0 in place.

You can also call a command with the --help flag appended to view command-specific information:

$ smithy build --help
Usage: smithy build [--help | -h]
                    [--debug] [--quiet] [--no-color]
                    [--force-color] [--stacktrace]
                    [--logging LOG_LEVEL]
                    [--config | -c CONFIG_PATH...]
                    [--no-config] [--severity SEVERITY]
                    [--allow-unknown-traits]
                    [--output OUTPUT_PATH]
                    [--projection PROJECTION_NAME]
                    [--plugin PLUGIN_NAME] [<MODELS>]

Builds Smithy models and creates plugin artifacts for each projection found in
smithy-build.json.
...

Throughout this post, we’ll be using the Smithy CLI on our example Weather model from the official Smithy documentation. You can follow along with the example or use your own models. Your local workspace should look like:

~/weather $ tree .
.
├── model
│   ├── weather.smithy
├── smithy-build.json

Building Models

Models are at the core of the Smithy language, and building models is the core functionality of the Smithy CLI. The build command performs validation and generates build artifacts, such as the JSON abstract syntax tree (AST), other models, code, and more.

Let’s build our basic example weather model:

~/weather $ smithy build model/

SUCCESS: Validated 240 shapes

Validated model, now starting projections...

──  source  ────────────────────────────────────────────────────────────────────
Completed projection source (240): weather/build/smithy/source

Summary: Smithy built 1 projection(s), 3 plugin(s), and 4 artifacts

Looking into the build artifacts (under weather/build/smithy/source), we’ll find several files based on our build configuration (smithy-build.json):

~/weather $ tree .
.
├── build
│   └── smithy
│       └── source
│           ├── build-info
│           │   └── smithy-build-info.json   -- build metadata (projections, validation, ...)
│           ├── model
│           │   └── model.json               -- JSON AST
│           └── sources
│               ├── manifest                 -- an inventory of the build's smithy models
│               └── weather.smithy           -- a copy of our example weather model
├── model
│   └── weather.smithy
└── smithy-build.json

In this case, no build behaviors outside of the defaults were configured in our smithy-build.json, so building our example weather model rendered only the JSON AST of the model. For more detailed information on configuring builds and build artifacts, check out the smithy-build guide.

Linting and Validating Models

Code validation tools, such as Checkstyle and SpotBugs, help developers avoid common pitfalls and bugs, while also ensuring that the code automatically adheres to standards of an organization. In Smithy, validators help prevent style issues and common mistakes, so that your team can focus on more important design considerations.

We can use a selector to choose which shapes to perform some custom validation on. As a best practice, we want to enforce documentation on all our operations, so we will add our selector to a validator in our model file:

// --- model/weather.smithy ---
$version: "2"

metadata validators = [{
    name: "EmitEachSelector"
    id: "OperationMissingDocumentation"
    message: "This operation is missing documentation"
    namespaces: ["example.weather"]
    configuration: {
        selector: """
            operation:not([trait|documentation])
        """
    }
}]

namespace example.weather
...

We’ll then run the validation on our weather model with the validate command:

~/weather $ smithy validate model/

──  DANGER  ────────────────────────────────────── OperationMissingDocumentation
Shape: example.weather#GetCity
File:  model/weather.smithy:46:1

45| @readonly
46| operation GetCity {
  | ^

This operation is missing documentation


──  DANGER  ────────────────────────────────────── OperationMissingDocumentation
Shape: example.weather#ListCities
File:  model/weather.smithy:92:1

88| // The paginated trait indicates that the operation may
89| // return truncated results.
90| @readonly
91| @paginated(items: "items")
92| operation ListCities {
  | ^

This operation is missing documentation
...

FAILURE: Validated 240 shapes (DANGER: 4)

With the ability to define custom validation rules like this, you can enforce a common standard for Smithy models, and be assured that best practices are upheld. For more information on validation in Smithy, please check out the Validation and Linting guides.

Differencing Models

If we make changes to our model, we’ll check for backward compatibility issues using the smithy diff command. For more information on the diff’ing process, see smithy-diff.

Let’s modify the time member in the GetCurrentTimeOutput shape in the example model. First, copy the model and rename the copied model (weather-old.smithy):

$ cp model/weather.smithy weather-old.smithy

Now, make the change to the shape in the “new” model file:

// --- model/weather.smithy ---
...
@output
structure GetCurrentTimeOutput {
    @required
    time: String
}
...

Now, let’s perform a compatibility check against this change:

~/weather $ smithy diff --old weather-old.smithy --new model/weather.smithy

──  DIFF  ERROR  ─────────────────────────────────────────── ChangedMemberTarget
Shape: example.weather#GetCurrentTimeOutput$time
File:  model/weather.smithy:138:5

136| structure GetCurrentTimeOutput {
···|
138|     time: String
   |     ^

The shape targeted by the member example.weather#GetCurrentTimeOutput$time
changed from smithy.api#Timestamp (timestamp) to smithy.api#String (string).
The type of the targeted shape changed from timestamp to string.

FAILURE: Validated 240 shapes (ERROR: 1)

We made a breaking change (ERROR) by changing the datatype of a shape that already had a previous definition. This is dangerous because of the break in backwards-compatibility between versions of your model. A client using your old model would no longer be able to safely call the GetCurrentTime operation. For more information on safely evolving your models, see the model evolution guide.

Querying Models

Selectors are a powerful way to query your model for different shapes. They can be used to build custom model validation logic through validators, or to define where certain traits can be applied. With the Smithy CLI, the process of working with and developing selectors is streamlined.

To query all of the shapes in our weather model, we can use the following statement:

~/weather $ smithy select --selector '[id|namespace = "example.weather"]' model/
example.weather#City
example.weather#CityCoordinates
example.weather#CityCoordinates$latitude
example.weather#CityCoordinates$longitude
...
example.weather#Weather

What if you want to find all undocumented operations in our model? We can answer this question by using the following statement:

~/weather $ smithy select --selector 'operation:not([trait|documentation])' model/
example.weather#GetCity
example.weather#GetCurrentTime
example.weather#GetForecast
example.weather#ListCities

You can iterate on this process as much as needed to answer questions about your model – once you have your selector, you can then use it in validation. For a greater understanding of Smithy selectors, please read through the Selectors guide.

Customizing Builds

One of the most powerful features of the build process is the extensibility. You can customize your build with the smithy-build.json file, adding projections or plugins as you desire. Let’s use the Smithy CLI to generate a client in TypeScript by configuring it to use the smithy-typescript code-generator. For a deeper dive into code-generation, check out the code generation guide.

Let’s make a new and more simple service model for demonstration purposes. Our workspace should have the following structure:

time
├── model
│   └── time.smithy      -- our new model file for the time service
└── smithy-build.json    -- a new smithy-build.json file

Our new model file, time.smithy, should have the following code:

// --- model/time.smithy ---
$version: "2"

namespace example.time

service Time {
    version: "0.0.1"
    operations: [GetCurrentTime]
}

/// An operation for getting the current time
@readonly
@http(code: 200, method: "GET", uri: "/time",)
operation GetCurrentTime {
    output := { 
        @required 
        @timestampFormat("date-time") 
        time: Timestamp 
    }
}

To generate the typescript client for the time service, our build configuration file should contain the TypeScript plugin and parameters to produce code for the time model:

// --- smithy-build.json ---
{
    "version": "1.0",
    "projections": {
        "source": {
            "plugins": {
                "typescript-codegen": {
                    "service": "example.time#Time",
                    "package": "@example/time",
                    "packageVersion": "0.0.1"
                }
            }
        }
    },
    "maven": {
        "dependencies": [
            "software.amazon.smithy:smithy-model:1.30.0",
            "software.amazon.smithy.typescript:smithy-typescript-codegen:0.14.0"
        ]
    }
}

The build will apply the typescript code-generator plugin to generate code, and will resolve the dependencies for the generator from Maven, as indicated by the maven section in the configuration. Let’s build our model and generate the code:

~/time $ smithy build model/

SUCCESS: Validated 378 shapes

Validated model, now starting projections...

[WARNING] Unable to find a protocol generator for example.time#Time: Unable to derive the protocol setting of the service `example.time#Time`
   because no protocol definition traits were present. You need to set an explicit `protocol` to generate in smithy-build.json to generate this service.
──  source  ────────────────────────────────────────────────────────────────────
Completed projection source (378): time/build/smithy/source

Summary: Smithy built 1 projection(s), 4 plugin(s), and 22 artifacts

Several TypeScript source files and configuration files are generated for the time client under the build/smithy/source/typescript-codegen directory in the workspace. A warning was printed because our Time service does not specify a protocol, but we can safely ignore this for demonstration purposes. Let’s take a look at a small snippet of the generated code in the build/smithy/source/typescript-codegen/src/Time.ts file:

// smithy-typescript generated code
import { TimeClient } from "./TimeClient";
import {
  GetCurrentTimeCommand,
  GetCurrentTimeCommandInput,
  GetCurrentTimeCommandOutput,
} from "./commands/GetCurrentTimeCommand";
import { HttpHandlerOptions as __HttpHandlerOptions } from "@aws-sdk/types";

export class Time extends TimeClient {
  /**
   * An operation for getting the current time
   */
  public getCurrentTime(
    args: GetCurrentTimeCommandInput,
    options?: __HttpHandlerOptions,
  ): Promise<GetCurrentTimeCommandOutput>;
  public getCurrentTime(
    args: GetCurrentTimeCommandInput,
    cb: (err: any, data?: GetCurrentTimeCommandOutput) => void
  ): void;
  public getCurrentTime(
    args: GetCurrentTimeCommandInput,
    options: __HttpHandlerOptions,
    cb: (err: any, data?: GetCurrentTimeCommandOutput) => void
  ): void;
  public getCurrentTime(
    args: GetCurrentTimeCommandInput,
    optionsOrCb?: __HttpHandlerOptions | ((err: any, data?: GetCurrentTimeCommandOutput) => void),
    cb?: (err: any, data?: GetCurrentTimeCommandOutput) => void
  ): Promise<GetCurrentTimeCommandOutput> | void {
    const command = new GetCurrentTimeCommand(args);
    if (typeof optionsOrCb === "function") {
      this.send(command, optionsOrCb)
    } else if (typeof cb === "function") {
      if (typeof optionsOrCb !== "object")
        throw new Error(`Expect http options but get ${typeof optionsOrCb}`)
      this.send(command, optionsOrCb || {}, cb)
    } else {
      return this.send(command, optionsOrCb);
    }
  }
}

From the preceding code snippet, we can observe the GetCurrentTime operation in our model was used in the code-generator to create a method, getCurrentTime, in the TypeScript client for the time service. If this were a real service, a customer could use this method to make the request to our service in their own TypeScript packages. Using just the Smithy CLI, we were able to build our simple time model, and generate some basic client code in TypeScript – to do this before, you would have needed to use Gradle and be familiar with the Gradle ecosystem to manage your project.

What’s Next?

Start using Smithy CLI today to streamline your experience when building models with Smithy. Smithy CLI makes it easy to build, validate, and rapidly iterate on your models.

We will be continuously making improvements to the CLI to enhance the developer experience, so tell us how you like using the Smithy CLI by leaving a comment or by contacting us on GitHub. Please don’t hesitate to create an issue or a pull request if you have ideas for improvements. Check out the Smithy documentation and user-guides over at smithy.io to learn more about how to leverage the full potential of Smithy.

About the author: