Extensible Software

Tristan M・ﾄリスタン — Mon, 10 Jul 2023 12:53:32 +0000

Good morning/afternoon/evening to yourself reader!

Preamble

I'd like to mention that this is my first post, and while it mightn't be very good, it wasn't written by an AI - so I think that has to count for something!
Although, that is exactly what an AI would say to garner your good favour...

I've been using Anki (mobile/desktop application to digitise flashcards and utilise SRS (Spaced-Repetition-Systems) to enhance the users ability to memorise aforementioned flashcards.

I really quite enjoy Anki, and owe my Japanese proficiency to itself after I ditched Memrise, but this is a blog for another day.

A feature of Anki, and some other applications that you may have used before, is the ability to download extensions that other users of the Anki community have created and made available out of the goodness of their hearts - and made available for all to download and enrich their Anki experience.

This got me thinking... How would someone make code like this?

Objective

I wanted to design something that would minimally fit the following requirements.

Run (this one is important)
Allow for the loading of zero to many files downloaded into a straightforward directory (think Skyrim's Mods directory).
Allow these extensions to interact with, and piggy back off main, and enrich the user experience.

Problems

I was not particularly concerned regarding step 1, writing software has been something that I've been doing for at least a little while (3 years).

For step 2, I'm accustomed to performing IO operations on files strewn about here and there across the filesystem, but these were always JSON/YAML configs, ENV Configurations, poorly formatted CSV files, et cetera. I'd never found myself in a position where I wanted much less thought it'd be a good idea to start running random scripts!

I was familiar with JavaScript's feature of eval which would look a little something like the following:

src: link to eval function example

I apologise for getting carried away with the example...

But this was different! This wasn't a line or 50, this was a whole file... surely there has to be a better way. There is!

Step 3 was also something that took at least a little bit of consideration. I needed to make sure that the extensions wouldn't come and unload themselves as soon as they finish executing, it's for times like these I'm grateful for my short stint at Unity game development!

Solutions

This took the longest, but that's because everything else kept breaking!
I achieved this by creating a function which simply looked for the extensions directory inside of the code's context, once inside we ~stole the jewels~ find all the .py files (could make this anything since it's plaintext files, maybe even .muggz?) and used a native library called importlib to swoop up the extensions code as if it was part of the program all along as an import!

src: link to above image

I provide all of these extensions with access to the central application state which we'll be discussing next!

To solve the dilemma of extensions doing their thing and going to buy milk when they ran out of lines to run - I opted to implement an Observer architectural pattern for the extensions to subscribe to pieces of state in the application, and register events that would fire in the event that a piece of state was changed.

This isn't a particularly advanced concept as it comes up quite intuitively when learning about front-end development, after all, an onclick event is merely the same concept.

In the interest of not having my code absolutely roasted to oblivion (though, you're welcome to do that also because I'm not infallible) this meant the creation of a few classes.

state_manager.py: holds a key:value dictionary of pieces of state. This is the middleman that all extensions and main use as a source of truth. Coordinates access to pieces of state.
emitter.py: an Observable (which would also be a valid name), this allows for the un/subscription of functions to entities. Also holds the logic for triggering all functions subscribed via an emit method.
state.py: inherits from emitter and is responsible for acting as an entry point for managing subscriptions, setter, and getter methods.

The code chunks for these classes are a little chunkier than those previous, and so I'm going to elect to not provide screenshots - if you're curious please don't hesitate to chuck out the git repo linked above.

Results

It worked! For the sake of testing and MVP main looks like the following:

Invoke StateManager to create a new store
Try to import a state.yaml file from parent directory to populate state
Create a state of name and set it's value to John
Load extensions

print_name.py

subscribe to 'name' in the state_manager (this will create state if not present, see print_age.py)
every time name is updated, print that the name was updated, and increment the name_changed state

from models.extension import Extension
from models.state_manager import StateManager

def print_name(state: StateManager):
    """
        This extension prints the name whenever it is changed.
        この拡張機能は、名前が変更されるたびに名前を表示します。
    """
    state.get("name").subscribe(lambda value: print("print_name extension: age was changed to", value))
    try:
        state.set("name_changes", state.get("name_changes").get() + 1)
    except:
        state.set("name_changes", 1)

ext = Extension("print_name", print_name)

print_age.py

Very structurally similar to print_name.py with one key difference.
Subscribe to 'age' state which does not exist YET.
This still subscribed to a state against key of 'age' however, the value is set to null, with a single subscriber (print_age.py)

from models.extension import Extension
from models.state_manager import StateManager

def print_age(state: StateManager):
    """
        This extension prints the age whenever it is changed.
        この拡張機能は、年齢が変更されるたびに年齢を表示します。
    """
    state.get("age").subscribe(lambda value: print("print_age extension: value was set to", value))
    try:
        state.set("age_changes", state.get("age_changes").get() + 1)
    except:
        state.set("age_changes", 1)

ext = Extension("print_age", print_age)

Change age to 30 (this invokes print_age.py's subscribed function)
Change name to Jane (invoking print_name's function)
print state_manager
save to yaml
end

Conclusion

This concludes my first blog post and short adventure into extensible software! It was a lot of fun, and a short respite from my current fullstack personal project that I'm looking forward to talking about when the MVP is finished!

Until then I hope you have a fantastic rest of your day/night!

DEV Community: Tristan M・ﾄリスタン