Reworking Lumberyard’s Entity Framework

Technology providers and studios have pumped an incredible resources into building technology that enables faster iteration, rapid feedback, early error detection, and enables team members of vastly different skills to work together to build high-quality gameplay.

As the engineer leading up some of our workflow efforts, I’ve had the joy of working with a team passionately focused on enhancing Lumberyard so that content creators can get creative as fast as possible. Today, I’ll share insight into Lumberyard’s new component entity system and its related workflows and some of the underlying technology. If you happen to be at GDC this week and didn’t catch our talk in the West Hall on Tuesday, just drop by the Lumberyard booth in the South Hall for juicy details and demos.

Something Everyone Uses

The entity system is a key construct in most engines and acts as the glue between engineers and content creators. Every team member interacts with the entity system in some way, so this was an obvious place for us to start our work to advance Lumberyard’s workflows and general usability. Prior to release, we worked with early customers using Lumberyard and leveraged our own veteran team’s experiences building games of many shapes and sizes. We captured mountains of feedback and set out to deliver something more familiar to designers and artists, and more flexible for engineers, to enable them to use Lumberyard’s vast feature set to build higher-quality content faster.

We built many fundamental engine systems comprising an ecosystem that has and will continue to drive major improvements in Lumberyard.

Component entity editor overview

Drag & drop entity creation & configuration

Component as Building Blocks

A component architecture intends to contain and hide complexity, and facilitate software growth and iteration. Components represent a game or engine’s features as bite-sized atomic pieces that work intuitively with minimal dependencies. There are many practical advantages of well-applied component entity architectures, including:

• Providing a common structure that engineers and content-creators can discuss and reason about together, enabling rapid design and development
• Promoting code re-use when compared to monolithic entity architectures, reducing maintenance costs
• Making deleting legacy code easier (like it should be) so you can minimize technical debt, reduce compile times, etc
• Allowing for entities to have predictable runtime performance cost – pay only for what you use
• Acting as an construct for making a large, source-available technology suite like Lumberyard easy to customize and extend to your own game-specific needs

One tenet of the Lumberyard’s new entity system is to keep entity life-cycle as simple as possible. In our experience, especially on large game projects, we’ve found complex entity life-cycles to be a major source of bugs and unnecessary complexity that makes it harder for your team to quickly evolve game features.

Lumberyard entities have two states: active, and inactive. That’s it. Once an entity is active, all components on the entity have a guarantee that the entity’s overall component make-up will not change. This simple rule makes writing reliable components straightforward, which means fewer errors. The lines are very clear as to when specific actions are possible. Additionally, Lumberyard components have the ability to express dependencies, requirements, and incompatibilities, in the form of services. The engine topographically sorts components within an entity based on these dependencies, to provide further mechanisms for component authors to minimize complexity (and bugs!).

An example of component services in the MeshCollider component:

    static void GetProvidedServices(AZ::ComponentDescriptor::DependencyArrayType& provided)
    {
         provided.push_back(AZ_CRC("ColliderService"));
    }
    static void GetRequiredServices(AZ::ComponentDescriptor::DependencyArrayType& required)
    {
        required.push_back(AZ_CRC("MeshService"));
    }

An example of simple activation and deactivation in the Mesh component:

    void MeshComponent::Activate()
    {
        MeshComponentRequests::Bus::Handler::BusConnect(m_entity->GetId());

        m_mesh.CreateMesh();
    }

    void MeshComponent::Deactivate()
    {
        MeshComponentRequests::Bus::Handler::BusDisconnect();

        m_mesh.DestroyMesh();
    }

Components that ship with the engine make diligent use of subscription-based, event-driven programming patterns. For example, entities and components do not implicitly receive per-frame tick messages. Instead, components use Lumberyard’s high-performance messaging system, called Ebuses, to communicate. Using the messaging system also minimizes dependencies between components, ensures they do not develop assumptions about how others operate internally, and avoids the need for brittle multi-stage initialization patterns.

    // Notify listeners that we've moved. This event bus supports any number of listeners.
    TransformNotificationBus::Event(GetEntityId(), &TransformNotificationBus::Events::OnTransformChanged, m_localTM, m_worldTM);

    // Listen for our parent's transform changes.
    TransformNotificationBus::Handler::BusConnect(m_parentId);

A Strong Foundation

We have also put a lot of thinking into Lumberyard’s aggressively-tested, generic, and high-performance serialization, reflection, and messaging systems. While we apply these throughout the engine, they’re used heavily by components. We want it to be trivial for component authors to express their class’s data makeup and editing behavior to others on the team who are likely to use their components. It’s necessary to have the ability to provide a simple and error-free editing experience for complex game or engine behaviors. Field ranges, validation functions, visibility filters, and many other attributes are available within the reflection markup to aid in building a robust component library. These features allow a sophisticated and robust editing experience with minimal code.

Undo/redo, cloning, saving/loading, a fully-cascading prefab system we call slices, and many other features rest on the shoulders of a high-performance object manipulation and serialization that interchangeably supports XML, JSON, and binary formats, and includes format-agnostic versioning. We continue to put large emphasis into squeezing every possible drop of performance out of these systems, especially given how fundamental they are to key elements of both the editor and runtime.

These are of course flexible and broadly applied elements of the core Lumberyard engine applied to more than just game entities. Given their versatility, we’re using them to power many of the engine’s features. You’ll see a lot happening in Lumberyard to ensure fundamental systems such as these are built well and stressed heavily.

An example of class reflection for serialization using AZ::Entity:

    serializeContext->Class<Entity>()->
      Version(2, &ConvertOldData)->
      Field("Id", &Entity::m_id)->
      Field("Name", &Entity::m_name)->
      Field("IsDependencyReady", &Entity::m_isDependencyReady)->
      Field("Components", &Entity::m_components);

An example of class reflection for editing using the Light Component (partial):

    ...
    DataElement("ComboBox", &LightConfiguration::m_lightType, "Type", "The type of light.")->
      Attribute("ChangeNotify", &LightConfiguration::LightTypeChanged)->
      EnumAttribute(LightType::Point, "Point"))->
      EnumAttribute(LightType::Area, "Area"))->
      EnumAttribute(LightType::Projector, "Projector"))->
      EnumAttribute(LightType::Probe, "Probe"))->
    DataElement(0, &LightConfiguration::m_onInitially, "On initially", "The light is initially turned on.")->
    ...

Cascading Prefabs, aka “Slices”

Lumberyard’s “Slices” are another key element of its entity foundation. Slices are a big topic and something we’re really excited about, so we’ll be doing a follow-up article where we can dive into the implementation, workflows, and further grand plans, but they’re definitely worth a mention here.

Slices are a cascading prefab system. Any number of entities with any arbitrary relationships and component configurations can be used to produce a slice asset; no hierarchy is required. Slices are assets, and can then be instantiated and optionally cascaded any number of levels. This allows you to finely tune and test entity setups with the ability to propagate enhancements and fixes throughout the slice hierarchy. A slice can contain any number of instances of other slices (aggregation), and any modifications (entities added/removed, components added/removed, fields changed), making them flexible for managing all entity data. For game entities in Lumberyard, everything is a slice, whether it’s a pre-configured AI squad or an entire level.

After instantiating a slice, it’s possible to modify it in any way and push the changes back to the underlying slice asset. You can change and push a single property change, or whole entities. This allows you to further refine and grow existing slices as you add behavior to the game.

Pushing instance changes to back to slice hierarchy

Entities & Components as a Generic Architecture

Compositional entity structures have been around for some time and applied in many ways.

We’re very actively addressing Lumberyard’s core engine DNA to make it extremely customizable and extensible. Game and system level components can be reflected to the engine from your own modules. All components are discovered by the engine through reflection and automatically become an integral part of Lumberyard. Your game components are available through all workflows and user interfaces, including the component palette.

System-level components augment the engine itself at a core level. Whether it’s your custom AI, game systems, or an entire physics integration, you can bring 3rd party or your proprietary technology and assets into the engine, or even contribute work back to share in the broader Lumberyard ecosystem. As we continue to aggressively modularize Lumberyard, it’ll be increasingly easier to disable elements of the engine you’re replacing or not using. Because Lumberyard modules integrate deeply with the build system, code in modules you don’t wish to include is completely stripped out of the build. You won’t even pay to compile the systems you don’t wish to use.

Intuitiveness Matters, Even For Experts

A technology or tool that isn’t intuitive creates a frustrating experience for new users. Often what defines an expert is someone who has invested enough time that they’ve mastered countless quirks and hidden “features.” Minimizing the time it takes to become an expert at using Lumberyard to build your projects is important, and we’re of course applying this philosophy in the design and implementation of Lumberyard’s component/entities.

We’re already working directly with game teams to ensure these workflows mature quickly. Intuitiveness, iteration times, early error detection, and performance are all getting tons of love to satisfy high-demand games in development right now, which accelerates our ability to get battle-tested and production-ready features to you in a state you can have confidence in.

There’s much more to talk about. Now that you’re a bit more familiar with some of Lumberyard’s new features, here are the topics we’ll be talking about in the coming weeks with much deeper technical detail:

• Generic reflection and serialization—rapidly edit and robustly manage, well, anything
• The nuts and bolts behind Ebuses—Lumberyard’s general-purpose high-performance messaging framework
• Slices—Lumberyard’s fully cascading prefab system
• Migrating legacy entity functionality to component entities

We’re excited to get your feedback on the next release of the new component entity system. If you have questions or great ideas for how you’d like to use this system, or would like to know more about its inner-workings, please don’t hesitate to let us know.

Grab Lumberyard, play with it, and tell us what you want to see next! Stay tuned for more juicy articles on how we’re constantly improving all aspects of Lumberyard.

Bill Merrill has been a professional software engineer in the game industry for 13 years, building experience in nearly all areas of game development and technology including pipelines, tools, gameplay systems, physics, networking, optimization, and especially AI and animation.