Building a Better Library Loader in C# - Part 3

In part 2, I went over the creation of a type at run-time by way of System.Reflection.Emit. We are able to define an interface we want to use, and our handy dandy LoadLibrary<T> method will take care of all the details for us. That, by itself, is pretty magical.

But we're not done. We need a few quality-of-life improvements.

Returning Strings

Let's go back to our SQLite interface.

interface ISqlite3
{
    int Open(string file, out IntPtr database);
    int Close(IntPtr database);
}

See how that Open method takes a string for one of its parameters? It really shouldn't work because System.String is a radically different data structure from the const char* that sqlite3_open expects. The first problem is that System.Char is 2 bytes, and the char in C is 1 byte. The second problem is that strings/arrays in .NET embed their length, and C strings use null-termination.

Thankfully, .NET foresaw this and took care of the issue for us. So, this methods works as you would expect! You can simply call sqlite3.Open("stuff.sqlite", out var db); without difficulty because your string is automatically converted to a null-terminated C string that your C library can actually handle.

However, what if we wanted a method that returns a string? Let's look at sqlite3_errmsg.

string Errmsg(IntPtr database);

I have bad news for you. If you call this, your program will crash. If it doesn't crash, it'll behave badly regardless. The nice type marshaling we had for values going in apparently does not apply to values coming out. So, how do we fix this? Well, since the C function returns a pointer (const char*), we need to capture it as a pointer.

IntPtr Errmsg(IntPtr database);

Great. How do I get a string out of that? Well, if you're using .NET Core, the answer is simple.

public static string GetString(IntPtr cString)
{
    return cString == IntPtr.Zero ? null : Marshal.PtrToStringUTF8(cString);
}

If you're not using .NET Core... shame on you. Start using .NET Core immediately! If you absolutely cannot use .NET Core (or a sufficiently high enough version of .NET Standard), you are going to have to parse the string yourself.

public static string GetStringTheHardWay(IntPtr cString)
{
    if (cString == IntPtr.Zero)
        return null;

    var bytes = new List<byte>();
    int n = 0;

    while (true)
    {
        byte b = Marshal.ReadByte(cString, n++);

        if (b == 0)
            break;

        bytes.Add(b);
    }

    return Encoding.UTF8.GetString(bytes.ToArray());
}

OK, so, now we have a way to read strings returned from C functions, but this is still too much work. Now, every time I want a string from my method, I have to call GetString(returnedValue). That's no good! I want to be lazier than that! So, let's revisit our code generator.

ILGenerator generator = methodBuilder.GetILGenerator();

for (int i = 0; i < parameters.Length; ++i)
    generator.Emit(OpCodes.Ldarg, i + 1); // Arg 0 is 'this'. Don't need that one.

if (Environment.Is64BitProcess)
    generator.Emit(OpCodes.Ldc_I8, ptr.ToInt64());
else
    generator.Emit(OpCodes.Ldc_I4, ptr.ToInt32());

generator.EmitCalli(
    OpCodes.Calli,
    CallingConvention.Cdecl,
    method.ReturnType,
    parameterTypes);

generator.Emit(OpCodes.Ret);

The Calli instruction places the C function's return value onto the stack, and the subsequent Ret instruction forwards it onto the caller. There is an opportunity here: we know, ahead of time, what the return type is. Why not just handle the conversion right here? If we know that the method on the interface wants to return string, we can take care of returning IntPtr internally and converting it before returning it.

// This should be stored outside the loop iterating over the interface methods.
MethodInfo getStringMethod = typeof(WhateverClassHasTheMethod).GetMethod(nameof(GetString));

ILGenerator generator = methodBuilder.GetILGenerator();

for (int i = 0; i < parameters.Length; ++i)
    generator.Emit(OpCodes.Ldarg, i + 1); // Arg 0 is 'this'. Don't need that one.

if (Environment.Is64BitProcess)
    generator.Emit(OpCodes.Ldc_I8, ptr.ToInt64());
else
    generator.Emit(OpCodes.Ldc_I4, ptr.ToInt32());

bool returnsString = method.ReturnType == typeof(string);

generator.EmitCalli(
    OpCodes.Calli,
    CallingConvention.Cdecl,
    returnsString ? typeof(IntPtr) : method.ReturnType,
    parameterTypes);

if (returnsString)
    generator.Emit(OpCodes.Call, getStringMethod);

generator.Emit(OpCodes.Ret);

Do you see what we did? The return type sent off to EmitCalli now has special treatment. If we ultimately want a string to come back, we capture an IntPtr first. Since that IntPtr value is on top of the stack after the Calli instruction, it is in the right position to serve as an argument for the call to GetString. How convenient! The call to GetString will place the string result onto the stack, and then our Ret instruction will return it to the caller.

Voila. Methods that return string work fine now.

Library Cleanup

Remember way back in part 1 when we talked about loading and unloading libraries? Our fancy-shmancy LoadLibrary<T> happily loads up the library but leaves us without a way to unload the library. That may not matter to many developers. It's quite normal for games to load up libraries, use them for the entire lifetime of the application, and then skip unloading them at the end. Regardless, I think it's best we at least offer up some way to unload these libraries as there are applications out there that need to load and unload libraries over time.

There are several approaches we can take here. I suggest leveraging IDisposable as it is idiomatic and brings several benefits along with it. The top benefit (in my mind) is the ability to put your library into a using block.

using (ISqlite3 sqlite3 = LoadLibrary<ISqlite3>("custom_sqlite3.dll", GetSqliteFunctionName))
{
    sqlite3.Open("my_data.sqlite", out IntPtr database);
    // ...
}

Another perk is that we can enforce this in the LoadLibrary<T> definition itself. Forgetting to mark your interface as IDisposable will result in a compiler error.

public interface ISqlite3 : IDisposable
{
    int Open(string file, out IntPtr database);
    int Close(IntPtr database);
}

public static T LoadLibrary<T>(
    string library,
    Func<string, string> methodNameToFunctionName)
    where T : class, IDisposable
{
    // ...
}

So, let's make a helper method to implement the Dispose method.

private static void CreateDisposeMethod(
    TypeBuilder typeBuilder,
    IntPtr libraryHandle)
{
    MethodBuilder methodBuilder= typeBuilder.DefineMethod(
        nameof(IDisposable.Dispose),
        MyMethodAttributes);
    typeBuilder.DefineMethodOverride(
        methodBuilder,
        typeof(IDisposable).GetMethod(nameof(IDisposable.Dispose)));

    Type ptrType = Environment.Is64BitProcess ? typeof(long) : typeof(int);
    ConstructorInfo intPtrConstructor = typeof(IntPtr).GetConstructor(new Type[] {ptrType});
    MethodInfo closeMethod = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ?
        typeof(WinLoader).GetMethod(nameof(WinLoader.FreeLibrary)) :
        typeof(UnixLoader).GetMethod(nameof(UnixLoader.Close));

    ILGenerator generator = methodBuilder.GetILGenerator();

    if (Environment.Is64BitProcess)
        generator.Emit(OpCodes.Ldc_I8, libraryHandle.ToInt64());
    else
        generator.Emit(OpCodes.Ldc_I4, libraryHandle.ToInt32());

    generator.Emit(OpCodes.Newobj, intPtrConstructor);
    generator.Emit(OpCodes.Call, closeMethod);
    generator.Emit(OpCodes.Pop); // Toss the returned int.
    generator.Emit(OpCodes.Ret);
}

As you can see, this method has to construct the IntPtr instance because there is no way to embed a constant IntPtr into the code. As I mentioned before, I recommend just using SharpLab in helping to determine what IL you want to emit. I'm no IL expert; I just write C# code for what I want to accomplish and study the resulting IL.

Just remember to call CreateDisposeMethod somewhere in your LoadLibrary<T> method, and you're gold. In part 4, we'll do more upgrades specific to game dev. :)

Comments

[DeafMan1983]

Hello I found wrong ".dll" under Unix:

string library = "path/to/custom/sqlite3.dll";
bool isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
IntPtr libraryHandle = isWindows ?
WinLoader.LoadLibrary(library) :
UnixLoader.Open(library, 1);

Replace with:
bool isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);
IntPtr libraryHandle = isWindows ?
WinLoader.LoadLibrary("path/to/custom/sqlite3.dll") :
UnixLoader.Open("path/to/custom/sqlite3.so", 1);

Because you use only *.dll o_O
If you use dynamic then it is file type as so for Linux/FreeBSD and dll for Windows

Thanks!

[Kelly Brown]

For the sake of simplicity, I've actually compiled my native libs to have a .dll extension even under Linux. Then my loader code can be the same for all platforms. I would never name a Linux library .dll if it were to become a system/shared library (as in /usr/lib), but if it's bundled with my own code for private usage, I don't see a problem.