Code Archeology: Crafting Lumberyard

Grab your weathered fedoras, we’re doing some code archaeology.

Engines evolve over time. The more people involved and the more projects it produces, the more an engine gets filled with all kinds of code. Code that fixes game-specific problems, code that is cleverly optimized, ultra-verbose code, code that looks like it participated in an obfuscated code competition, great code, and orphaned code.

At Amazon we strive to be Earth’s most customer-centric company, and Lumberyard’s commitment to game development is no exception. As we advance the Lumberyard engine, we continually ask ourselves, “Is this the best code we can deliver to our customers?” If the answer is not a definite yes, then we know we have some work to do. The code we want to deliver should be readable, consistent, extensible, correct, performant, logical, peer reviewed, documented, modern, and useful. This means we study the code and identify areas we can improve without sacrificing or hindering existing functionality. So how do we make sense of it all? Just as it is in field archaeology, we dust off one piece at a time.

Let’s examine one code artifact we uncovered in our code spelunking: IEntityProxy.

What is an entity proxy?

An entity proxy is a special type of entity component. By themselves, they are not very different from other components, the primary difference is in how and when they are used by entities.

Entities can host any number of components. Entity proxies are a type of component that is managed separately from the other components for a few reasons. One reason is for quick access; normal components are not meant to be available for random access, but entity proxies are stored in a map, which allows us to grab them using a value from the EEntityProxy enum as the key. Another reason is to control their lifetime. Many entity proxies are fairly core-level engine components (rendering, physics, audio), and may have inter-dependencies. As a result, during shutdown, we need to be mindful of the order in which we release them to prevent getting into the situation in which a proxy has been released while we still needed it.

So what’s the problem?

We want Lumberyard to be a very modular, component-based engine that is easy and intuitive to extend. To this end, we determined that the IEntityProxy system represents a layer of complexity and tight coupling to engine systems that we could not efficiently maintain over time. For example, in its existing design, adding a new low level component would require the addition of a new value into the EEntityProxy enum in IEntityProxy.h, which is a low level interface. This addition would affect the serialization order for all other proxies. But more egregiously, it would cause any files that depend on IEntity to be compiled. Depending on the number of entities in the code, this could take a significant amount of valuable time.

In fact, the entity proxy system does have an extensibility feature. There is a slot reserved for a “user” entity proxy, so why not use that? One problem with this design is that it hinders our ability to reuse code, a lot of code becomes special case for “user” proxies, and we lose the ability to easily distinguish between different user proxies, if there were any, without adding another layer of information. Another big problem with it is the fact that there is only one slot for a user proxy.

We made the decision to fold IEntityProxy and replace it with a component based system in which any component can inherently be accessed in O(1), that is by design extensible with very little boilerplate code and that does not involve modifying any centralized header file.

This work would simplify the existing code base and bring it closer in line to the goal of having an entirely new component system and reduce the effort of porting existing engine components into Lumberyard components when the new framework became available.

Throughout our exploration, we found many interesting paths. Some were clear, others posed a challenge, and a few were fraught with peril. IGameObjectExtension is also a specialized type of component used as a mechanism to bring component support to CGameObjects. Down the IGameObjectExtension path there is a lot of code that we did not delve into in this journey. We marked our map and we’ll come back to it another day.

As we explored further, we found some functions that existed for convenience, depending on programmer preference. The following two examples are equivalent:

IEntityRenderProxy *pRenderProxy = (IEntityRenderProxy*)pEntity->GetProxy(ENTITY_PROXY_RENDER); //  Matching lines: 54
IEntityRenderProxy *pRenderProxy = (IEntityRenderProxy*)pEntity->GetRenderProxy(); // Matching lines: 29

We all have ways we prefer to read and write code, so finding multiple approaches to the same problem is not uncommon. We replaced these two functions with a single way to retrieve all components with the goal of having a very clear, concise framework.

Early on we decided that an enum based type identifier was not extensible enough. Instead, we opted to replace it with a type based identifier, which meant we could leverage templates to retrieve the component of the desired type.

IComponentRenderPtr renderComponent = m_pEntity->GetComponent<IComponentRender>();

In addition to the syntactical change, we reduced the number of functions used to get components to just this one and we took the opportunity to make component pointers managed using std::shared_ptr<>—in fact they already were, but they were not consistently used.

Having moved away from a centralized enum for component types, we were now able to specify their type directly within each component. This has the added benefit of reducing compilation times when new components are added since we no longer modify a central header file that was pulled in wherever we included entity code.

A minimal component now looks like this:

class ExampleComponent : public IComponent
{
   DECLARE_COMPONENT_TYPE("ExampleComponent", 0xB2B34F13F1A04686, 0xBB49697AAD456275);
public:
};
DECLARE_COMPONENT_POINTERS(ExampleComponent);

Components could now exist entirely within .cpp files! In fact, the unit tests we wrote for this system exist entirely in source files.

A few more vines to swing from

So far we felt we had done some pretty tangible improvements to the code, and as our journey continued deeper and deeper we found yet another opportunity in the shape of this pattern:

if (!GetRenderProxy())
    CreateProxy(ENTITY_PROXY_RENDER);

We understood the need for the creation of proxies, but we wanted to simplify it. We absorbed the burden of verifying if the component exists ourselves and provided this function:

IComponentRenderPtr renderComponent = m_pEntity->GetOrCreateComponent<IComponentRender>();

This is very similar to the way we already retrieved our components. Then, almost by accident, we stumbled across this little guy tucked away in two different precompiled headers on the game side code:

inline IEntityProxy* GetOrMakeProxy( IEntity *pEntity,EEntityProxy proxyType ) { ... }

As the name implies, this function will check if the entity already has a proxy of that type. If it does, it returns the proxy, which gives us O(1) access. If it doesn’t, it will create a proxy, register into the proxy map, and then return it.

This is pretty much the same thing we did. This was good, it felt like validation that what we were thinking was a good idea. We dug around a bit more to see how often it was used and where we would need to go and replace it.

Find all "GetOrMakeProxy"... ... Matching lines: 3

Three results. That’s the declaration in each of the precompiled headers and the one use case in a file called RadioChatterModule.cpp. We didn’t know GetOrMakeProxy existed for a while. It was buried in precompiled header for game code and not in the engine side so that fact alone limited its scope.

For Lumberyard we placed this utility function as a member of IEntity, which makes it accessible throughout the code base, both engine and game-side. This gave us the ability to take advantage of it in more places:

Find all "GetOrCreateComponent"... ...Matching lines: 138

We went from three calls to GetOrMakeProxy to over a hundred calls to GetOrCreateComponent. The achievement here is that we now have a consistent way to get and/or create proxies and slightly reduced the amount of code needed to use components. Of course we now had the ability to grep the code and quickly find all the places where components were being lazily created. Sometimes, it’s the little things that matter.

Cue the giant, unusually round boulders!

We tried to be careful archaeologists. We gently dusted off our work area, we skillfully disarmed the booby traps and documented our findings (it’s not science unless you write it down). So, what did we break?

Well…

We experienced difficulties with the lifetime and order of destruction of components. We learned that some components may reference other components in their destruction and so the order must be preserved. One mistake we made was to inadvertently reverse the order of destruction of components. This lead to components expecting other components to still exist at the time of their own destruction.

Another mistake we made was in communication. In the course of our changes we noticed the pattern used for determining entity event priority:

virtual ComponentEventPriority GetEventPriority( const int eventID ) const { return ENTITY_PROXY_LAST - const_cast<IEntityProxy*> (this)->GetType(); }

Noticing how the priority was always LAST – ProxyType, we thought we might as well just flip the event priority ordering and return the value of the priority:

So before:

enum EEntityProxy
{
    ENTITY_PROXY_RENDER,
    ...
    ...
    ENTITY_PROXY_USER,
    ENTITY_PROXY_LAST
};
virtual ComponentEventPriority GetEventPriority( const int eventID ) const { return ENTITY_PROXY_LAST - const_cast<IEntityProxy*> (this)->GetType(); }

While in the new implementation we would simply do:

struct IComponentAudio : public IComponent
{
    ...
    ComponentEventPriority GetEventPriority(const int eventID) const override { return EntityEventPriority::Audio; }
    ...
};

With the caveat that we reversed the enum ordering:

namespace EntityEventPriority
{
    enum
    {
        User = 1,
        ...
        ...
        Render,
        GameObjectExtension
    };
}

So in the previous implementation the audio proxy event priority would be:

return ENTITY_PROXY_LAST - ENTITY_PROXY_AUDIO; // 15 - 3 results in 12

In the present implementation it is:

return EntityEventPriority::Audio; //  the value of EntityEventPriority::Audio is 12

The entity event priority calculation was equivalent with the added benefit that the code was slightly more readable, and had one less subtraction operation. However, we did not propagate the explanation of these changes to Lumberyard users. There was a gap between our deployment of these changes and when other teams began the process of integrating their own work. This is when they encountered this change and had questions about what our intentions were. Once they saw our intentions were benign, they lowered their spears and invited us to feast with them. They introduced us to this wonderful beverage called beer in their native language.

In conclusion

“When you do things right, people won’t be sure you’ve done anything at all.” – Futurama, S3E20

We are crafting Lumberyard to be a powerful game engine that will allow game developers to make all kinds of games. We believe that in order to achieve this, we need to find opportunities to make the code scalable, readable and generally easy to use and maintain. Replacing IEntityProxy with a component based system brings us closer to this goal allowing developers to create new components without so much overhead.

For newcomers to Lumberyard, this is a bit of history and trivia. We hope you found the insight useful. For those familiar with Lumberyard’s origins, we hope this conveys some of our thoughts and vision for the future.