First and most importantly this project progressed enough that it now needs a catchy, searchable name. I have a couple candidates that are fine though they don't feel right. There are Latin words pyrope which is a kind of red or rust colored gem though inferior to a ruby gem; and punicus which sounds better though it has multiple meanings. Along with the color bright red it may also relate to old Carthage and the Roman bloodlust for Carthaginian genocide which isn't what I want to evoke.
Now the less important stuff
(dev.to embedded gifs aren't functioning at the moment)
https://dev-to-uploads.s3.amazonaws.com/i/7afo799od7nrmgegw90v.gif
I've made a tremendous amount of progress in a few days it's hard to talk about where things are at. I'll detail the most interesting thing I came across the past few days, a few quick hits and then where I'm going next.
I'm using the Dear ImGUI library for creating the GUI.
Dear ImGui
(This library is available under a free and permissive license, but needs financial support to sustain its continued improvements. In addition to maintenance and stability there are many desirable features yet to be added. If your company is using Dear ImGui, please consider reaching out.)
Businesses: support continued development and maintenance via invoiced sponsoring/support contracts
E-mail: contact @ dearimgui dot com
Individuals: support continued development and maintenance here. Also see Funding page.
It's a tiny easily embedded interface library written in an orthodox C++, or C+, style. That is, almost entirely functions and structs without any of the C++ nightmares. Im means immediate mode GUI, in contrast to retained mode. The primary difference between the two is what controls the state of the interface. Most people are used to working with something that looks like a retained mode GUI. The user of a retained mode interface notifies an underlying system of changes. When notifying GUI system of changes the GUI library changes internal data structures and rebuilds lists of commands for drawing the interface. Working with the browser DOM is a great example of retained mode style. We tell the browser presentation engine when things change by modifying the DOM elements, not by telling the browser how to render a page directly.
Immediate mode style GUI does away with that state and hands the user of the API complete control of how to draw all the elements. Instead of notifying a system when state changes, with an immediate mode design the GUI code you write is instead executed linear sequence every frame. It's best shown with an example, here from the Dear ImGUI demonstration
if (ImGui::BeginMenuBar())
{
if (ImGui::BeginMenu("Menu"))
{
ShowExampleMenuFile();
ImGui::EndMenu();
}
if (ImGui::BeginMenu("Examples"))
{
ImGui::MenuItem("Main menu bar", NULL, &show_app_main_menu_bar);
ImGui::MenuItem("Console", NULL, &show_app_console);
ImGui::MenuItem("Log", NULL, &show_app_log);
ImGui::MenuItem("Simple layout", NULL, &show_app_layout);
ImGui::MenuItem("Property editor", NULL, &show_app_property_editor);
ImGui::MenuItem("Long text display", NULL, &show_app_long_text);
ImGui::MenuItem("Auto-resizing window", NULL, &show_app_auto_resize);
ImGui::MenuItem("Constrained-resizing window", NULL, &show_app_constrained_resize);
ImGui::MenuItem("Simple overlay", NULL, &show_app_simple_overlay);
ImGui::MenuItem("Manipulating window titles", NULL, &show_app_window_titles);
ImGui::MenuItem("Custom rendering", NULL, &show_app_custom_rendering);
ImGui::MenuItem("Documents", NULL, &show_app_documents);
ImGui::EndMenu();
}
if (ImGui::BeginMenu("Tools"))
{
ImGui::MenuItem("Metrics", NULL, &show_app_metrics);
ImGui::MenuItem("Style Editor", NULL, &show_app_style_editor);
ImGui::MenuItem("About Dear ImGui", NULL, &show_app_about);
ImGui::EndMenu();
}
ImGui::EndMenuBar();
}
Reading this top to bottom you know exactly what's going on. Two points of interest. One, the if() calls around ImGUI functions generally mean 'if this element is visible'. If that element is completely obscured in some way it doesn't execute that code path. Amazingly simple! The other interesting point is Dear ImGUI elements usually take a boolean parameter. This is a reference to an address of memory representing if the element should be displayed or disabled. The interface for creating a new window is ImGui::Begin("Ruby Eval", &ruby_eval_open). The second parameter becomes false when the user closes the window. We can re-open the window whenever we want by setting ruby_eval_open to true.
(This pattern works fantastically in C and comes back to bite us when we're in Ruby land. Discussions for another day)
Casey Muratori describes the differences and benefits of an immediate mode in great detail here
Setting aside HN style nonsense on the computational efficiency or wastefulness of this pattern, the benefit to this is we have full control over what we want to draw when and the clear simplicity of it.
Back to making the GUI work with Ruby. It's a straightforward process making an mruby class in C in which calling the methods on the class in Ruby execute C code. For now I'm generally mapping the ImGUI functions to a GUI module one to one.
In a World...
The main program loop is in C. It looks something like this, abbreviated
while(!done) {
process_window_messages();
begin_new_frame();
mrb_load_string(mrb, "World.render");
render_frame();
}
The important bit here is mrb_load_string. It's essentially an eval into the mruby state. Surrounding the call processes incoming Windows messages, sets up the next frame, then renders the frame. Presently World.render is the entry point for each frame from the Ruby side. World is another simple construction. It maintains a list of all the open windows. When we call World.render it iterates over all of the open windows calling render on each.
module World
@open_windows = []
def self.add_window window
raise unless window.respond_to? :render
@open_windows.push window unless @open_windows.include? window
end
def self.remove_window window
@open_windows.delete window
end
def self.render
# TODO see if current active window respond_to? menu_bar,
# otherwise render this
GUI.main_menu do
GUI.menu("Tools") do
GUI.menu_item "Class Browser", "Ctrl+I" do
ClassBrowser.open
end
GUI.menu_item "Open Transcript", "Ctrl+T" do
Transcript.open
end
GUI.menu_item "Open Stats", "" do
Stats.open
end
end
end
@open_windows.each { |window| window.render }
rescue => ex
puts ex
end
end
When you want to open a window, call World.add_window thing. Closing it is World.remove_window thing. Simple and straight forward.
Implementing a render method is a simple set of ImGUI calls. For example, this is the full Transcript, a global console-like thing where you can print stuff to
module Transcript
@lines = []
def self.add obj
@lines.push obj.to_s
end
def self.clear
@lines = []
end
def self.open
World.add_window self
end
def self.close
World.remove_window self
end
def self.render
GUI.new_window 'Transcript' do
@lines.each { |line| GUI.text line }
end
end
end
This all works fine and I was reasonably satisfied with it so far. Until one time I left the program open for several hours I noticed the program acquired multiple gigabytes of working memory. It usually allocates about 300kb total. Classic memory leak somewhere. Getting to the heart of why is a long saga detailed in this short Github exchange.
Effective ways on running Ruby code from within C without leaking Proc objects?
#5001
Short summary
mrb_load_string, related functions and the typical run mruby code from C patterns create an RProc object on every call which is never marked for GC. There's an easy reproduction case with a simple C program
#include "stdio.h"
#include "ext/mruby.h"
#include "ext/mruby/compile.h"
#include "ext/mruby/gc.h"
int main() {
mrb_state *mrb = mrb_open();
for(int i = 0; i < 100000; ++i) {
mrb_load_string(mrb, "nil");
if(i % 100 == 0) {
printf("%zi\n", mrb->gc.live);
}
}
return 0;
}
Longer explanation
I'm working on embedding mruby in a GUI. The C portion of program sets up all the initial rendering, then passes control to the mruby VM, then renders the frame after the VM finishes. I call the ruby VM exactly once each frame from C with one entry point. My main program loop looks roughly like
while(!done) {
setup_frame();
mrb_load_string(mrb, "World.render");
render_frame();
}
And this works wonderfully! The fast C code path does the intensive graphics calculations and rendering, and I have one entry point which kicks off processing the VM every 'tick' with World.render Leaving out a lot of unnecessary surrounding context here to get at the heart of the matter.
After being so happy for a couple days on how easy this was going I left my program overnight, coming back the program taking up multiple gigabytes of memory where it usually only takes a few hundred kb. Tell tale memory leak of some kind. I did a lot of digging within my own C code as that's the most likely answer and came up with nothing.
Looking at the live object count every frame I noticed the live object count never decreases. I started digging in the Ruby code thinking I created object islands and came up with nothing. Finally I changed my loop to do mrb_load_string(mrb, "nil"); and much to my surprise the object count increased linearly infinitely just as before when the mruby VM shouldn't do anything at all!
Digging some more into ObjectSpace while simply calling nil every frame I found that T_PROC objects grow indefinitely. After a few minutes of running
{:TOTAL=>73728,
:FREE=>206,
:T_OBJECT=>2,
:T_CLASS=>59,
:T_MODULE=>10,
:T_ICLASS=>14,
:T_SCLASS=>75,
:T_PROC=>73285,
:T_ARRAY=>2,
:T_HASH=>2,
:T_STRING=>23,
:T_EXCEPTION=>2,
:T_ENV=>44,
:T_DATA=>3, 23=>1}
Looking at the definition of mrb_load_string and others related it it's usually the same sequence of function calls: creating a new parser context, parsing the string, generating a proc object, run the code in the vm, free the parser and context. I replicated this replacing mrb_load_string in my loop
mrbc_context* ctx = mrbc_context_new(state->ruby_state);
mrb_parser_state *parser = mrb_parse_string(state->ruby_state, "World.render", ctx);
RProc *proc = mrb_generate_code(state->ruby_state, parser);
mrb_value retval = mrb_vm_run(state->ruby_state, proc, mrb_top_self(state->ruby_state), 0);
mrb_pool_close(parser->pool);
mrbc_context_free(state->ruby_state, ctx);
And realized there are the infinitely growing Proc objects created. Digging through old issues I had the idea of calling mrb_gc_arena_save/mrb_gc_arena_restore before and after the loop... and this works! The object count stabilizes over time. Hurray, problem solved!
However I'm not completely satisfied. Going back to the old 'principle of least surprise' I found it highly surprising that the typical patterns used in functions like mrb_load_string leak objects.
Some questions
- It feels like a mistake that functions like
mrb_load_stringand it's relatives leak Proc objects on every call. Should we do a gc save/restore within these functions? - Is there a better way to invoke a Ruby method from C without creating a new RProc object?
- Alternately, is there a way we could mark the RProc used for execution as black immediately after running it in the VM so it becomes GC'd next cycle? This feels like it's crossing too many boundaries however.
The short answer is this innocent looking C call
mrb_load_string(mrb, "World.render");
Which it turns out any C function named mrb_load_* creates an RProc object in the current mruby state. However, that RProc is not garbage collected at any point! For reasons I sort understand this document details the situation. However it feels surprising that the mrb_load_* function calls don't automatically protect against that situation.
Going back to the main loop, we call mrb_load_string every frame, yielding to the World object for rendering all the windows. That generates an RProc object in the VM which the garbage collector never collects. We do the at least 60 times a second, so we're leaking 60 RProc objects a second. A basic Ruby object is tiny, 36 bytes or so which is why this behavior was not noticeable.
I've since added basic GC stats into the interface so I can observe such things
https://dev-to-uploads.s3.amazonaws.com/i/jam8uak08klab6mwbvum.gif
Next up
As it stands currently this is primarily a C program driving the mruby state as shown by the main program loop is in C. I want to flip this around where the main program loop is in Ruby which calls back to C code when necessary. I want the C main function to look something like
int main() {
setup_state();
mrb_load_string(mrb, "GUI.start");
cleanup();
return 0;
}
And then the Ruby side looking something like
class GUI
def self.start
while is_running do
process_window_events
begin_next_frame
render
present_frame
end
end
end
I think this will take a significant amount of work and I know it will pay off later. Reading through past mruby issues and recommendations by matz the general solution to most of the integration problems is 'push what you're doing into the mruby VM instead of C'.
I didn't think of moving the loop into ruby this way until I saw this demonstration of a madman using mruby for creating a Sega Dreamcast game. This combination of esoteric things, homebrew Dreamcast development and mruby, make me quite happy it exists.
mrbtris: Sample game for Sega Dreamcast written in Ruby
mrbtris is a simple game for the Sega Dreamcast which is written in Ruby as an proof-of-concept of using the mruby implementation on the Sega Dreamcast. This project was written by @yujiyokoo.
This project is built on the top of KallistiOS (KOS), which is the low-level library used to interact with the Sega Dreamcast hardware. Usually, programs written for the Sega Dreamcast are in C/C++, this project aims to demonstrate the use of Ruby source code targeting the Sega Dreamcast.
This project aims to provide a simple example of how to use KallistiOS (KOS) API and mruby together.
Demonstration
Below you may find a video of this game running on the real hardware (early version)
TODO
- Make an
mrbgemfor the Dreamcast specific things - Create unit tests
- Use more Sega Dreamcast features
Building
mrbtris uses KallistiOS (KOS) and mruby…
Top comments (0)