Introducing Smithy IDL 2.0
The AWS Smithy team is happy to announce the 2.0 release of the Smithy Interface Definition Language (IDL). This release focuses on improving the developer experience of authoring Smithy models and using code generated from Smithy models. It contains numerous new features, such as reduced nulls and optional types in generated code, custom default values, mixins to reduce model duplication, resource properties to improve consistency across operations, dedicated enumeration shapes, and syntax improvements.
Smithy is a protocol-agnostic IDL and set of tools for generating clients, servers, documentation, and other artifacts. It’s purpose-built for code generation, can be extended with custom traits, and enables automatic API standards enforcement. Smithy is Amazon’s next-generation API modeling language, based on our experience building tens of thousands of services and generating SDKs.
Fewer nulls in generated code
One goal of Smithy IDL 2.0 is to reduce the amount of nullable properties in generated code. With the addition of Rust, Kotlin, and Swift as supported programming languages in AWS SDKs, it became apparent that we needed a better approach for defining nullability in Smithy models to make generated code in these languages more idiomatic. However, we also had to balance this desire against the business requirement of allowing services to safely evolve their models over long periods of time (sometimes decades).
Prior to Smithy 2.0, Smithy code generators essentially treated every structure member as nullable. This was because: 1) nullability was somewhat confusing and people modeling structures didn’t realize that default values were provided by only some of the shapes targeted by members, and 2) the
@required trait could be backward compatibly removed at any time because it was considered the removal of a constraint.
Consider the following Smithy IDL 1.0 example:
In the above example:
Messagehas a default value of
MyBooleanisn’t marked with the
@boxtrait. These semantics were inherited from Smithy’s predecessor within Amazon, and many teams get confused by how these semantics work because it’s a confusing kind of “action at a distance” from the member definition.
messagemember is nullable because the
@requiredtrait can be removed at any time. This allowed teams using Smithy 1.0 to remove the
@requiredtrait from a member if they ever needed to due to changing business requirements. As such, using the
@requiredtrait to influence code-generated types was expressly prohibited by the specification.
Smithy IDL 2.0 introduces custom default values for structure members. The previous example is written in Smithy IDL 2.0 as:
With the introduction of default values in Smithy IDL 2.0, we realized that we can still provide almost the same level of model evolution flexibility while still reducing nullability in generated code. In Smithy IDL 2.0, if the
@required trait is ever removed from a member, the member has to be provided a default value. This new affordance means that code generators can safely use the
@required trait and default values to generate types that contain non-nullable properties. For example, if a structure member is
@required, then accessing the member will always return a value. The same is true for members that have default values: they will always return a value even if one wasn’t explicitly provided by the end-user.
Without these updates, languages like Rust that explicitly model nullability in their type systems would generate optional accessors. This would cause users to have to unsafely dereference most structure members.
The nullability improvements in Smithy IDL 2.0 simplify how nullability is modeled, still provides developers the ability to evolve models over time if they need to remove the
@required trait, and explicitly model default values of structure members rather than only relying on documentation.
Please note that implementing these Smithy IDL 2.0 nullability semantics in Smithy code generators and AWS SDKs is still a work in progress, but they will be incorporated before they made are generally available.
You can read more about member nullability in the Smithy documentation about structure member optionality.
Mixins to reduce model duplication
When authoring a Smithy model, the same members often need to be re-defined in related structures. Manually copy-pasting parts of shapes in a model is not only tedious; it’s also error prone and can lead to unintentional drift between shapes over time as different parts of a service are updated. Mixins are a kind of model-time copy-paste that eliminates this kind of model duplication. You can learn more about mixins in the Smithy documentation for mixins.
Defining resources in Smithy 1.0 often requires a significant amount of duplication across inputs and outputs. The following example describes a single operation in Smithy 1.0:
Utilizing mixins for this API reduces the repetition and removes surface area for inconsistencies in the model (especially as more operations are added). Here is an updated version of the above model, using resource properties (explained later in the blog post) and mixins:
Mixins can be used with other shape types as well, like enumerations of various platform options, operations with a consistent set of errors, and services that share common operations like this example:
Smithy models can define resources to communicate the “things” in an API. Resources in Smithy 1.0 only included operations and identifiers. This allowed services to define the resources in their APIs and ensure that operations are provided the right identifiers, but the lack of validation to ensure that resources expose properties consistently across operations can lead to usability issues. For example, it was easy to define a service with a
PutFoo operation that accepts an
idList member in its input while a
GetFoo operation returns a corresponding member as
ids. This kind of inconsistency is undesirable because it puts a usability burden on end users to determine that these members mean the same thing.
Smithy 2.0 adds the ability to define the
properties of a resource. With this new declaration, Smithy will automatically detect when a team accidentally uses the wrong property name in the input or output of a resource based operation.
Each property defines a name and the shape it targets. The input and output of resource operations are limited to members that match these property names and targets (with a few caveats explained in the specification). The new
@notProperty trait can be used to indicate that a member of an instance operation isn’t a resource property, like a
dryRun boolean or an idempotency token.
Smithy 2.0 also introduces a new feature called member elision to refer to resource identifiers, resource properties, or mixin members. To use member elision, add a
$ character before a member name and omit the target of the member so it is inherited. This syntax eliminates the need to redefine the shape targeted by the referenced component, and makes it clear that the target is inherited.
Note: In order to refer to resource properties or identifiers with member elision syntax, bind the resource to a structure using the following
Dedicated enumeration shapes
Smithy 1.0 previously defined enumerations using an
@enum trait on a string shape. This resulted in several issues: the syntax used to define enums was verbose, enum values weren’t members so removing enum values from different projections (or views) or a Smithy model required special-casing, and the metadata associated with an enum value duplicated traits like
Smithy 2.0 introduces the
intEnum shapes and deprecates the
intEnum are used to represent a fixed set of string and integer values. You can learn more in the Smithy documentation for enum shapes and intEnum shapes.
Making enums dedicated shape types improves the modeling experience and makes their use with other parts of Smithy more consistent. Enum values are now members of enum shapes, meaning documentation, tags, and other properties now use the standard Smithy traits instead of additional properties of enum value definitions. Since enums are now shapes with their values as members, they’re affected by model transformations without needing to write specialized code.
This example shows how enums were defined in Smithy 1.0:
This changes significantly when using an
enum shape in Smithy 2.0, utilizing traits and documentation comments instead.
intEnum shape type provides the ability to define enumerated integers, rather than just strings. This enables compatibility with various serialization formats that use integers for enums rather than strings.
To see the complete list of changes for this release, check out the Smithy repository changelog.
- Already using Smithy 1.0 and ready to migrate to Smithy 2.0? Get started with the IDL 1.0 to 2.0 migration guide.
- If you are new to Smithy and IDL 2.0 and want to explore more, take a look at the Smithy documentation and watch this introductory video from Michael Dowling, one of Smithy’s creators.
- For feature requests, bug reports, contributions, or feedback of any kind, head over to the Smithy GitHub repository.