SObjectizer Tales - Epilogue

What a journey it has been!

Never did I imagine crafting 30 episodes. Initially, I planned for a modest 15 or perhaps 20 episodes, but my willingness for sharing led me down an unexpected path of creativity. Despite the likelihood of limited readership due to its length and niche topic, I am ok with that. Yet, above all, I embarked on this endeavor for my own growth and self-expression. It began last August, during a well-deserved break before my holiday adventures commenced, and remained with me for more than six months.

The series was for me like a boat to traverse the river, but now that I’ve reached the other shore, it must be left behind as I cannot bear its weight on my back to continue my journey on foot.

Before diving into this final installment, I want to express my gratitude once more to Yauheni Akhotnikau. Beyond being a reviewer, I believe I’ve found a friend in him. Throughout each episode, Yauheni not only provided feedback but also engaged in meaningful exchanges about life. I am thankful for the chance to connect with him on a deeper level.

This final post is structured as follows: first, we offer a brief recap of the series; next, we present an overview of missing features that were not covered throughout the series; finally, we give some suggestions for initiating work with SObjectizer and the paradigm in general.

TL;DR: Rewind

The series focused on developing calico, a program tailored for acquiring and processing images from a camera – specifically, the default webcam on our machine. Utilizing SObjectizer, we encapsulated logic within self-contained actors (agents) that communicated via messages, following the publish-subscribe pattern. SObjectizer managed actor and channel lifecycles, data distribution, and agent logic execution on threads, employing dispatchers to meet our requirements.

Collaborating with colleagues, we incrementally added new features, exchanged ideas, and expanded our knowledge. Each post explored a specific aspect of calico and introduced corresponding features of SObjectizer that facilitated its development, occasionally allowing for broader reflections. Ultimately, calico emerged as a versatile environment where various and original functionalities could be achieved by combining actors through channels.

First, we discussed the implementation of 4 image producers:

• image_producer: implementing a tight acquisition loop employing the “blocking” camera API;
• image_producer_recursive: implementing a message passing-styled loop with the “blocking” camera API;
• image_producer_callback: employing the “non-blocking” camera API;
• virtual_image_producer: implementing a message passing-styled loop with a simulated “blocking” API, reading images from the disk;

Then, we delved into the development of an extensive set of processing agents for various functionalities, occasionally touching upon broader themes as they arose. Here’s a breakdown:

Core:

• face_detector: detects faces in frames and overlays them (or nothing if none detected);
• image_cach: caches images until a certain dimension is reached;
• image_resizer: resizes images;
• image_saver: Saves images to disk;
• service_facade: enables external processes to receive images and to send commands via gRPC;
• static and dynamic processing pipelines: demonstrates “pipes and filters” and “routing slip” patterns;
• stream_detector: provides signal-based logic to detect streaming start and stop.

UI:

• image_viewer and live_image_viewer: display images (and also their counterparts that run on the main thread only);
• remote_control: sends commands using keyboard (and also its counterpart that runs on the main thread only).

Monitoring:

• error_logger: logs any image acquisition error to console;
• fps_estimator: estimates and logs the frame rate of any channel;
• image_tracer: logs images received from any channel;
• stream_heartbeat: logs the uptime of a stream from any channel;
• telemetry_agent: demonstrates the usage of runtime telemetry.

Most of these agents support input channels, and some also support output channels. For example, image_viewer receives images from an input channel but does not output to any channel. Conversely, image_resizer takes input images and sends resized frames to an output channel.

When we say “channels” we mean “message boxes”, which are the main abstraction in SObjectizer for transmitting messages and signals.

This design offers powerful features. Firstly, encapsulation is achieved, as agents communicate solely through messages without direct knowledge of each other. This allows for versatility, such as feeding agents with images from any source, including test cases, for example. Secondly, the design promotes extensibility by enabling the addition of new agents without altering existing ones. Introducing a new feature may only require the development of a new agent. Additionally, the paradigm’s flexibility extends to thread binding, as agents aren’t inherently tied to specific threads. Users have the flexibility to assign agents to worker threads as needed, even dynamically. Finally, this model simplifies concurrent code, as agents ideally don’t share state. Synchronization is handled by the framework solely on message queues, eliminating the need for explicit synchronization mechanisms.

In calico, the instantiation and combination of agents are hard-coded. However, supporting dynamically-loaded configurations to generate processing pipelines at runtime would be relatively simple to implement and it’s exactly what I have developed in my real scenario.

Furthermore, during the series we explored broader topics such as testing, measuring performance, shutdown prevention, binding agents to threads (dispatchers), and various high-level design considerations and patterns. Concluding our series, we discussed aspects of SObjectizer that we found less favorable or areas where we envisioned potential improvements.

At this point, let’s briefly touch upon a few features that weren’t explored throughout the series.

Missing features carousel

We now briefly explore some features that weren’t covered in the series because they weren’t necessary for my purposes. However, they may offer solutions to specific problems and should be taken into account for future projects. I’ve categorized these features into two groups: ordinary and advanced.

In the ordinary category:

Message Delivery Tracing aims to debug an application built on top of SObjectizer. In essence, it logs the primary stages of the message delivery process, allowing visibility into whether there is a suitable subscriber with the corresponding event handler.

Deadletter Handlers is a shorthand for handling the same message in the same way from different states of the same agent. For example, instead of this:

void so_define_agent() override
{
    so_subscribe_self()
      .in(st_first)
      .in(st_second)
      .in(st_third)
      .event(&some_agent::on_some_message);
}

We can write this:

void so_define_agent() override
{
    so_subscribe_deadletter_handler(so_direct_mbox(), &some_agent::on_some_message);
}

Unique Subscribers Message Box – recently brought from so5extra to SObjectizer – is a special MPSC message box, permitting subscription from multiple agents as long as they subscribe to different messages. This enables each handler to handle mutable messages. This capability is especially valuable in processing pipelines. For instance, consider a data-processing scenario with multiple processing stages, each represented as an agent, and a coordinating agent managing the processing through mutable messages: the manager sends a message to the first stage agent, receives the result, then forwards a message to the second stage agent, and continues in this manner.

Due to the necessity of mutable messages, only MPSC mailboxes are viable. However, this requires the manager to know MPSC mailboxes for each processing stage, which can be inconvenient. While it would be simpler to have a single mailbox for all outgoing mutable messages, the standard MPMC mailbox permits subscription from different agents but does not support mutable message exchange. Here is where unique subscribers message boxes come into play.

Custom Direct Message Box enables programmers to construct an agent’s direct message box, instead of using the default one provided by SObjectizer.

In the advanced category, we find low-level features that cover uncommon use cases:

Custom Worker Threads enables to replace underlying thread implementation used by dispatchers.

Event Queue Hooks enables the creation of specialized wrappers around event queues, which can serve various purposes such as tracing and collecting runtime statistics.

Environment Infrastructures allows for the specification of the threading strategy within SObjectizer’s environment. By default, there are at least three background threads: one for the default dispatcher, another for timers (the timer thread), and an additional thread for completing deregistration operations. Furthermore, SObjectizer provides the option to choose a single-threaded environment, which can be either thread-safe or not, tailored for specific use cases.

Subscription Storage is a fundamental data structure within SObjectizer, responsible for storing and managing subscription information for each agent. Depending on the use case, agents may create varying numbers of subscriptions, ranging from a few to potentially thousands. Therefore, the choice of data structure is crucial. SObjectizer offers flexibility in selecting different storage options, including vector-based, map-based, hash-based, etc. By default, the strategy is adaptive: for agents with few subscriptions, a small and fast vector-based storage is utilized. However, as the number of subscriptions grows, the agent automatically switches to a more suitable, albeit more resource-intensive, map-based storage solution.

Locks Factory is an advanced tool for selecting different event queue synchronization schemas. The functionality extends beyond data protection to include notification of consumers about the arrival of new events. While the default synchronization scheme is efficient under heavy loads, it may not be optimal for certain load profiles. SObjectizer addresses this issue by enabling the specification of another simpler locking schema that is based on mutexes and condition variables that might be more lightweight for some use cases.

Enveloped Messages allows the transmission of messages or signals enclosed within a special object known as an “envelope”. These envelopes can carry additional information and execute specific actions upon delivery of the message/signal to a receiver. This is considered a low-level feature primarily intended for use by SObjectizer’s developers or by those seeking to extend its functionality.

Where to go from here

Are you interested in utilizing SObjectizer? Are you seeking inspiration for practicing with this paradigm? In this concise section, we’ll provide you with some ideas to explore.

To begin with, consider utilizing calico as your personal gym, as its development journey may still have more to offer. There are several areas left for further enhancement. These include introducing missing tests, implementing dynamic pipeline generation as previously mentioned, adding new agents for currently unsupported operations, supporting multiple cameras or devices, and more. The possibilities for expansion are virtually limitless.

Additionally, consider starting your own project. Identify a use case and experiment with leveraging SObjectizer and its supported paradigms to develop it. Concurrency might be simpler if you think of it as individual actors exchanging messages. Seeking ideas?

• interactive chat
• simple games
• message broker
• video streaming application

Another valuable exercise involves reviewing others’ code. You can explore the “by example series” in the official wiki, which demonstrates more advanced use cases of SObjectizer. Additionally, there are various open-source projects that leverage SObjectizer, offering inspiration and insight into its practical applications, such as:

• arataga, a socks5+http/1.1 proxy server;
• Shrimp, a HTTP-server that provides shrunk images generated from a specified set;
• mosquitto_transport, an experiment of writing SObjectizer-based wrapper around mosquitto library;
• LabNet, a server to control hardware connected to RaspberryPi over Ethernet.

Furthermore, consider rethinking an existing concurrent application through the lens of SObjectizer and its capabilities. Does it fit well? What functionalities are lacking? Remember that the SObjectizer community is readily available to assist you.

Finally, I summarize a thoughtful remark by Yauheni regarding the adoption of SObjectizer. He emphasizes that the main issue with SObjectizer is its lack of widespread experience and visibility on the internet, making it prone to misuse and resulting in poor outcomes. He advises gaining experience with SObjectizer before using it extensively in serious development. For new projects, he suggests that the lack of experience is less critical as initial development phases allow for experimentation and quick fixes. Yauheni highlights the risk of integrating SObjectizer into large, established projects without sufficient expertise and suggest two safer approaches: first, using SObjectizer for auxiliary tasks like writing emulators to gain experience, and second, encapsulating SObjectizer functionalities within isolated subsystems with familiar interfaces to ensure flexibility in case of integration issues. This strategy allows for safe adoption and potential changes if integration problems arise.

Last but not least, whether you have questions or seek design advice, don’t hesitate to open an issue for support and discussion. Sometimes, as happened to me several times, such questions turn into new features.

See you next time!

Have you read the entire series or maybe just some episodes?! I’d love to hear your thoughts!

Would you like to see more content like this? What aspects did you enjoy the most and which ones the least about the series? Please reach out via email or leave a comment here with your feedback!

Thank you and see you next time!

---

Thanks to Yauheni Akhotnikau for having reviewed this post.