loading...

Lumberyard EBus from Rust

jeikabu profile image jeikabu Originally published at rendered-obsolete.github.io on ・8 min read

Rust in Lumberyard (4 Part Series)

1) Rust in Lumberyard 2) Lumberyard Editor Plugin in Rust 3) Lumberyard EBus from Rust 4) Lumberyard Asset Builder in Rust

One of the architectural changes Amazon made to CryEngine is the introduction of the EBus (Event Bus):

a general-purpose communication system that Lumberyard uses to dispatch notifications and receive requests

Ebuses are a key participant in the component and gem systems and used throughout the engine. Their indirection and de-coupling make them an ideal candidate for integrating additional languages in Lumberyard development.

For further details about ebus, see:

EBus Primer

Let’s take a look at using the TickBus:

#include <AzCore/Component/TickBus.h>

class MyEbusHandlerComponent
    : public Component
    , public AZ::TickBus::Handler
{

private:
    void Activate() override
    {
        // Connect to the bus
        TickBus::Handler::BusConnect();
    }

    void OnTick(float deltaTime, ScriptTimePoint time) override
    {
        // Handle tick event
    }
};

// Send a message on `TickRequestBus`
float frameDeltaTime = 0.f;
TickRequestBus::BroadcastResult(frameDeltaTime, &AZ::TickRequestBus::Events::GetTickDeltaTime);

To subscribe to an ebus and receive published events:

  1. Derive from EBus<T>::Handler
  2. Call EBus<T>::Handler::BusConnect()

Broadcast() and BroadcastResult() are used to send messages (requests) to an ebus.

Down the Template Rabbit Hole

Let’s take a look at what’s going on by unraveling the innocuous looking:

TickBus::Handler::BusConnect();

TickBus is defined in Code/Framework/AzCore/AzCore/Component/TickBus.h:

/**
* The EBus for tick notification events.
* The events are defined in the AZ::TickEvents class.
*/
typedef AZ::EBus<TickEvents> TickBus;

Ok, so we’re working with an EBus. EBus is defined in Code/Framework/AzCore/AzCore/EBus/EBus.h:

template<class Interface, class BusTraits = Interface>
class EBus
    : public BusInternal::EBusImpl<AZ::EBus<Interface, BusTraits>, BusInternal::EBusImplTraits<Interface, BusTraits>, typename BusTraits::BusIdType>
{
    //...

Because only one template parameter is specified, both Interface and BusTraits template parameters are the same type (i.e. EBus<TickEvents, TickEvents>).

EBusImpl is defined in Code/Framework/AzCore/AzCore/EBus/BusImpl.h:

template <class Bus, class Traits>
struct EBusImpl<Bus, Traits, NullBusId>
    : public EventDispatcher<Bus, Traits>
    , public EBusBroadcaster<Bus, Traits>
    , public EBusBroadcastEnumerator<Bus, Traits>
    , public AZStd::Utils::if_c<Traits::EnableEventQueue, EBusBroadcastQueue<Bus, Traits>, EBusNullQueue>::type
{
};

//...

template <class Bus, class Traits>
struct EBusBroadcaster
{
    /**
    * An event handler that can be attached to only one address at a time.
    */
    using Handler = typename Traits::BusesContainer::Handler;
};

It inherits from a few things, but the one we care about is EBusBroadcaster which defines Handler.

So, to summarize what we have figured out so far:

//TickBus::Handler::BusConnect();
EBusImplTraits<_, TickEvents>::BusesContainer::Handler::BusConnect();

EBusImplTraits is defined in BusImpl.h

template <class Interface, class BusTraits>
struct EBusImplTraits
{
    //...

    /**
    * Contains all of the addresses on the EBus.
    */
    using BusesContainer = AZ::Internal::EBusContainer<Interface, Traits>;

That means we now know:

//TickBus::Handler::BusConnect();
//EBusImplTraits<_, TickEvents>::BusesContainer::Handler::BusConnect();
AZ::Internal::EBusContainer<_, TickEvents>::Handler::BusConnect();

Code/Framework/AzCore/AzCore/EBus/Internal/BusContainer.h:

// Default impl, used when there are multiple addresses and multiple handlers
template <typename Interface, typename Traits, EBusAddressPolicy addressPolicy = Traits::AddressPolicy, EBusHandlerPolicy handlerPolicy = Traits::HandlerPolicy>
struct EBusContainer
{
public:
    //...

    using Handler = IdHandler<Interface, Traits, ContainerType>;

Thing is, IdHandler doesn’t define BusConnect(). The comment gives you a clue about what’s going on- specialization/partial specialization based on AddressPolicy and HandlerPolicy from the trait (i.e. TickRequests).

TickEvents in TickBus.h:

/**
* Interface for AZ::TickBus, which is the EBus that dispatches tick events.
* These tick events are executed on the main game thread. In games, AZ::TickBus
* dispatches ticks even if the application is not in focus. In tools, AZ::TickBus 
* can become inactive when the tool loses focus.
* @note Do not add a mutex to TickEvents. It is unnecessary and typically degrades performance.
*/
class TickEvents
    : public AZ::EBusTraits
{
    //...

EBusTraits in EBus.h:

struct EBusTraits
{
public:
    /**
        * Defines how many handlers can connect to an address on the EBus
        * and the order in which handlers at each address receive events.
        * For available settings, see AZ::EBusHandlerPolicy.
        * By default, an EBus supports any number of handlers.
        */
    static const EBusHandlerPolicy HandlerPolicy = EBusHandlerPolicy::Multiple;

    /**
        * Defines how many addresses exist on the EBus.
        * For available settings, see AZ::EBusAddressPolicy.
        * By default, an EBus uses a single address.
        */
    static const EBusAddressPolicy AddressPolicy = EBusAddressPolicy::Single;

Therefore the specialization we’re interested in is EBusContainer<_, _, EBusAddressPolicy::Single, EBusHandlerPolicy::Multiple>:

// Specialization for single address, multi handler
template <typename Interface, typename Traits, EBusHandlerPolicy handlerPolicy>
struct EBusContainer<Interface, Traits, EBusAddressPolicy::Single, handlerPolicy>
{
public:
    using ContainerType = EBusContainer;
    using IdType = typename Traits::BusIdType;

    //...

    using Handler = NonIdHandler<Interface, Traits, ContainerType>;

Now we finally know what TickBus::Handler is:

//TickBus::Handler::BusConnect();
//EBusImplTraits<_, TickEvents>::BusesContainer::Handler::BusConnect();
//AZ::Internal::EBusContainer<_, TickEvents>::Handler::BusConnect();
NonIdHandler<_, TickEvents>::BusConnect();

Look at definition of NonIdHandler in Code/Framework/AzCore/AzCore/EBus/Internal/Handlers.h:

/**
* Handler class used for buses with only a single address (no id type).
*/
template <typename Interface, typename Traits, typename ContainerType>
class NonIdHandler
    : public Interface
{
public:
    using BusType = AZ::EBus<Interface, Traits>;
    //...

    void BusConnect();
    void BusDisconnect();

Sure enough, there’s the declaration for BusConnect(). But, what about the definition? That’s back in Ebus.h:

template <typename Interface, typename Traits, typename ContainerType>
void NonIdHandler<Interface, Traits, ContainerType>::BusConnect()
{
    typename BusType::Context& context = BusType::GetOrCreateContext();
    AZStd::scoped_lock<decltype(context.m_contextMutex)> contextLock(context.m_contextMutex);
    if (!BusIsConnected())
    {
        typename Traits::BusIdType id;
        m_node = this;
        BusType::ConnectInternal(context, m_node, id);
    }
}

this is the AZ::TickBus::Handler-derived handler instance from which we called BusConnect() (e.g. in Activate()). This looks like the end, let’s just pinch this off: it grabs a lock and calls EBus<>::ConnectInternal() (also in Ebus.h), which calls Connect() on EBusContainer instance member (back in BusContainer.h), and adds our handler to the list of handlers.

All that template shenanigans to add a pointer to a list. Good times.

Rust-ified

All of that was very interesting, but how can we call it from Rust? Bindgen doesn’t support several “advanced” C++ techniques like template functions, partial template specialization, and traits templates.

This results in a few problems for us:

  1. No way to resolve types

    • As we found above, because of template specialization TickBus::Handler::BusConnect() is NonIdHandler<_, TickEvents>::BusConnect(). But Bindgen only generates bindings for the base template:
    pub type EBusContainer_Handler<Interface> = root::AZ::Internal::IdHandler<Interface>;
    
  2. There’s no functions to bind (link) to

    • Almost everything is defined via templates in header files; mostly inlined and not externally visible symbols

The simplest approach is wrapping the needed routines in “plain” C functions like we did to deal with Lumberyard Editor IPlugin:

#include <AzCore/Component/TickBus.h>
#include <AzCore/Script/ScriptTimePoint.h>

extern "C" {
    void TickRequestBus_BroadcastResult_GetTimeAtCurrentTick(AZ::ScriptTimePoint& results)
    {
        AZ::TickRequestBus::BroadcastResult(results, &AZ::TickRequestBus::Events::GetTimeAtCurrentTick);
    }
}

In Rust:

extern "C" {
    pub fn TickRequestBus_BroadcastResult_GetTimeAtCurrentTick(results: *mut root::AZ::ScriptTimePoint);
}

Simple and effective, albeit tedious. But we’re not done yet.

Look at the definition of ScriptTimePoint in Code\Framework\AzCore\AzCore\Script\ScriptTimePoint.h:

class ScriptTimePoint
{
public:
    AZ_TYPE_INFO(ScriptTimePoint, "{4c0f6ad4-0d4f-4354-ad4a-0c01e948245c}");
    AZ_CLASS_ALLOCATOR(ScriptTimePoint, SystemAllocator, 0);

    ScriptTimePoint()
        : m_timePoint(AZStd::chrono::system_clock::now()) {}
    //...
}

And system_clock::now() in Code\Framework\AzCore\AzCore\std\chrono\clocks.h:

class system_clock
{
public:
    //...
    static time_point now() { return time_point(duration(AZStd::GetTimeNowMicroSecond())); }
    //...
};

All of this is inlined leaving us with no way to correctly initialize ScriptTimePoint in Rust.

There’s a few different ways to deal with this:

extern "C" {
      #[link_name = "\u{1}?now@system_clock@chrono@AZStd@@SA?AV?$time_point@Vsystem_clock@chrono@AZStd@@V?$duration@_JV?$ratio@$00$0PECEA@@AZStd@@@23@@23@XZ"]
      pub fn system_clock_now() -> root::AZStd::chrono::system_clock_time_point;
  }
  impl system_clock {
      #[inline]
      pub unsafe fn now() -> root::AZStd::chrono::system_clock_time_point {
          system_clock_now()
      }
  }
  • Wrap the function and export it from C++:
extern "C" {
      time_point system_clock_now() {
          return system_clock::now();
      }
  }

Then in Rust:

extern "C" {
      pub fn system_clock_now() -> time_point;
  }
  • Re-implement the helper in Rust:
impl AZStd::chrono::system_clock {
      pub fn now() -> AZStd::chrono::system_clock_time_point {
          unsafe {
              let time = AZStd::GetTimeNowMicroSecond();
              let duration = AZStd::chrono::duration::from_duration_rep(time);
              AZStd::chrono::time_point::from_time_point_duration(duration)
          }
      }
  }

  impl<Rep> AZStd::chrono::duration<Rep> {
      pub fn from_duration_rep(val: Rep) -> Self {
          Self { m_rep: val, _phantom_0: PhantomData }
      }
  }

  impl<Duration> AZStd::chrono::time_point<Duration> {
      pub fn from_time_point_duration(val: Duration) -> Self {
          Self { m_d: val, _phantom_0: PhantomData }
      }
  }

At the moment, some combination of the first two seems like the right choice:

  1. Requires least amount of code to write/debug/maintain
  2. If original C++ code is removed/renamed, we’ll get a compile time error to update the wrapper
  3. Leverage bindgen to automatically generate Rust bindings

Solution

First, we create wrappers in a C++ static library to access from Rust:

// RustAz.h
namespace AZStd {
    namespace chrono {
        system_clock::time_point system_clock_now()
        {
            return system_clock::now();
        }
    }
}
// RustAz.cpp
#include "RustAz.h"

RustAz/wscript to produce RustAz.lib:

def build(bld):
    bld.CryEngineStaticLibrary(
        target = 'RustAz',
        #...
    )

Build, and dumpbin /symbols <root>BinTemp\win_x64_vs2017_profile\Code\Framework\RustAz\RustAz\RustAz.lib confirms it is an externally visible symbol defined in the library:

287 00000000 SECT113 notype () External | 

?system_clock_now@chrono@AZStd@@YA?AV?$time_point@Vsystem_clock@chrono@AZStd@@V?$duration@_JV?$ratio@$00$0PECEA@@AZStd@@@23@@12@XZ 

(class AZStd::chrono::time_point<class AZStd::chrono::system_clock,class AZStd::chrono::duration< __int64,class AZStd::ratio<1,1000000> > >__ cdecl AZStd::chrono::system_clock_now(void))

Next, process RustAz.h with bindgen to generate the Rust bindings:

pub mod AZStd {
    pub mod chrono {
        extern "C" {
            #[link_name = "\u{1}?system_clock_now@chrono@AZStd@@YA?AV?$time_point@Vsystem_clock@chrono@AZStd@@V?$duration@_JV?$ratio@$00$0PECEA@@AZStd@@@23@@12@XZ"]
            pub fn system_clock_now() -> root::AZStd::chrono::system_clock_time_point;
        }
    }
}

Ebus-in’

From Rust we can now make a request via the ebus and receive a simple result:

unsafe {
    use lmbr_sys::root::{AZ, AZStd};
    let mut current_time = AZ::ScriptTimePoint { m_timePoint: AZStd::chrono::system_clock_now() } ;
    lmbr_sys::TickRequestBus_BroadcastResult_GetTimeAtCurrentTick(&mut current_time);
    info!("GetTimeAtCurrentTick {:?}", current_time);
}

I must admit, the prospect of having to re-write and manually maintain extensive bindings is daunting and dissuades me from wanting to continue pursuing this. However, there doesn’t seem to be a better option until bindgen gets support for more than the most trivial C++.

Rust in Lumberyard (4 Part Series)

1) Rust in Lumberyard 2) Lumberyard Editor Plugin in Rust 3) Lumberyard EBus from Rust 4) Lumberyard Asset Builder in Rust

Posted on by:

Discussion

markdown guide