DEV Community: Kelly Brown

Introducing Jawbone Sockets

Kelly Brown — Sun, 01 Jun 2025 18:27:40 +0000

Hi! Sorry I'm so wildly inconsistent about blogging. I've just been hard at work on a lot of things. :)

I ranted previously about the awful state of .NET socket libraries for game devs. I originally developed my new socket library inside the confines of my Jawbone lib, but the project evolved enough and sparked enough interest from third parties that I decided it was worth spinning off into its own focused project.

Introducing... Jawbone.Sockets!

Be sure to take a tour to become familiar with the API design. The sockets perform zero allocation (beyond the socket creation itself), and they indeed perform better than System.Net sockets, but the more time I spend with this library, the more I just enjoy the drastically simplified design. Span<T> is the heart and soul of everything in this library.

IpAddress Internals

I do want to draw attention to a few things that are not in the tour. It took me a while to settle on how I wanted to structure IpAddressV4 and IpAddressV6. I had a recurring issue where I wanted to reinterpret the data in the address depending on the context. IPv4 addresses are sometimes bytes but sometimes just a whole UInt32. IPv6 addresses are sometimes bytes, sometimes an array of UInt16, and sometimes an array of UInt32. Initially, I tackled this by simply exposing a bunch of methods that converted the address to a respective span, but I later noticed the recent addition of inline arrays. I was not a fan at first. They seemed clunky and hard to use, but once I discovered that they implicitly convert to spans, I knew it was the way forward.

So, IpAddressV4 and IpAddressV6 are both just union structs. They have multiple ways to access the underlying data.

var target = new IpAddressV4(10, 0, 0, 1);
target.DataU8[3] = 15; // Changed to 10.0.0.15.

uint raw = target.DataU32; // Capture entire address.

// Fancy way of initializing 2001:db8:4006:812::200e
var v6 = IpAddressV6.FromHostU16([0x2001, 0xdb8, 0x4006, 0x812], [0x200e]);

// Captures the 0x4006 in network order!
ushort segment = v6.DataU16[2];

// Set every byte to 0xab via span!
v6.DataU8[..].Fill(0xab);

// Walk each 32-bit chunk.
foreach (uint chunk in v6.DataU32)
  Console.WriteLine(chunk);

This union approach made implementing other aspects of the structs trivial. Constructing, parsing, formatting, hashing, and equating all hugely benfited from this flexibility. (And none of it required inheritance, allocations, or dynamic dispatch.)

These IP address types are exactly as big as they need to be, so they pack tightly. This makes them good for use cases such as keys in a dictionary.

Generic Addressing

Splitting IP addresses into two concrete types obviously presents a few problems. What if an application wants to support both IPv4 and IPv6? Does that mean there has to be two complete implementations of everything?

Well yes... but no. At least, no one should be typing out two complete implementations. Both IpAddressV4 and IpAddressV6 implement IIpAddress<T>, so as long as the communications subsystem is generic, it can handle both no problem.

public sealed class MultiplayerSystem<T> where T : unmanaged, IIpAddress<T>
{
  private readonly Dictionary<T, PlayerSessionInfo> _sessions = [];
  private readonly IUdpSocket<T> _serverSocket;

  public MultiplayerSystem(IUdpSocket<T> serverSocket)
  {
    _serverSocket = serverSocket;
  }

  // ...
}

Much of the functionality provided by IIpAddress<T> is possible thanks to static virtual members interfaces. Any generic code is still free to parse text via T.Parse(...) or target localhost by way of T.Local. Naturally, there are non-static methods available as well such as address.IsLoopback (where address is T).

Despite everything I've said, there is also a storage type called IpAddress. It's another struct that simply uses an enum to track whether it's holding an IPv4 address or IPv6 address. This is handy for code that really doesn't care, but it's worth noting that this container doesn't interact with sockets at all. The sockets expect specific address types.

Conclusion

For now, the lib only targets .NET 9. I may add support for .NET 8 if support doesn't end by the time I polish everything up for a 1.0 release. I'm using this lib in a couple projects of my own. Once I have a substantial application to demonstrate this package's viability, I'll feel better about going GA.

If you're interested, take the tour, grab the latest package on NuGet, and tell me what you think!

Jawbone Networking Motivation

Kelly Brown — Sat, 10 Jun 2023 17:34:32 +0000

Some quick context: Jawbone is my game library.

Motivation

The big secret game project I'm working on right now is multiplayer and uses UDP for its networking. Naturally, at the start of all this, I looked inside .NET itself since there are classes for working with UDP, but I found myself unable to stomach the offerings there. I have a couple requirements of the code I work with in game dev.

First, allocations must be kept to a minimum. In C#, this usually means I wanna see two things: structs and spans. Where possible, I want to keep things off the heap: structs sit nicely on the stack in local contexts. From there, if I must use the heap, I want data packed tightly, and structs sit nicely side-by-side in an array or in the body of another class/struct. And lastly, I want to slice my arrays freely. If I need to work with some portion of an array, spans let me do that without copying the whole array. Also, circling back the first point in this paragraph, if I must allocate memory, I don't want my library code doing that invisibly. Let me allocate my own byte arrays once and simply reuse them by way of Span<byte> and ReadOnlySpan<byte>.

Second, dynamic dispatch must be kept to a minimum. This impacts both performance and readability. Where possible, I don't want my code branching like crazy when performing a straightforward operation. I don't need polymorphism most of the time. I don't need last-minute runtime decisions most of the time. I frequently know the truth up front and would like to compile that truth into my engine.

So, how do the UDP classes in .NET measure up to these standards? Well, they're not great. Please note that I understand that classes in the core libraries aspire to be all things to all people and are dealing with decades of legacy code/usage by this point. I'm not attacking the people who made this. Regardless, I feel there is value in explaining exactly what could be improved for contexts like mine.

System.Net.Sockets.UdpClient

Let's look at the main contender: UdpClient. Right out of the gate, I'm not thrilled with the design.

First off, I'm not going to touch ReceiveAsync or BeginReceive/EndReceive. I love async C# but not in game dev. So, that just leaves me with Receive (docs).

public byte[] Receive(ref IPEndPoint? remoteEP)

See the problem? Well, there are two problems, but we'll get to the other problem later. The method allocates and returns a byte array. I find it strange there is no overload that accepts a Span<byte> yet to let me fill my own buffer. The Send method has already received some love.

public int Send(
    ReadOnlySpan<byte> datagram,
    IPEndPoint? endPoint)

You know what's funny? Take a look at the source code for Receive.

int received = Client.ReceiveFrom(_buffer, MaxUDPSize, 0, ref tempRemoteEP);
remoteEP = (IPEndPoint)tempRemoteEP;

// because we don't return the actual length, we need to ensure the returned buffer
// has the appropriate length.

if (received < MaxUDPSize)
{
    byte[] newBuffer = new byte[received];
    Buffer.BlockCopy(_buffer, 0, newBuffer, 0, received);
    return newBuffer;
}
return _buffer;

It writes to an interim buffer anyway. C'mon! Just let me pass in a Span<byte>.

System.Net.Sockets.Socket

Oh well. That's not gonna work. So, let's move onto the next contender: Socket. This looks much better. It's lower level. It provides nearly direct access to the socket itself. And right out of the gate, it solves the primary allocation problem (docs).

public int ReceiveFrom(
    Span<byte> buffer,
    ref EndPoint remoteEP)

Sending is fine too.

public int SendTo(
    ReadOnlySpan<byte> buffer,
    EndPoint remoteEP)

So, now what? We solved most of the allocation problem... but now we are contending with the lovely address/endpoint classes, which also introduces dynamic dispatch.

System.Net.IPAddress

As cool as it is that .NET supports a royal cacophony of address families, all we really care about is IPv4 and IPv6. Of the two, IPv4 is arguably the most important even today, but supporting both is straightforward. (Seriously, think about just how many games would break if IPv4 ceased being an option.)

So, that brings us to IPAddress, which is the catchall datatype for socket addresses either IPv4 or IPv6.

Strike 1: Heap objects

So, this class has decided that all instances must live in the heap. Thanks for that. Oh, but it gets better.

/// <summary>
/// This field is only used for IPv6 addresses. A null value indicates that this instance is an IPv4 address.
/// </summary>
private readonly ushort[]? _numbers;

So, an IPv6 address is two heap objects. Wonderful. It just gets worse as you go down the method list. Requesting the address in bytes allocates a byte array for you. Again, I know this is a casualty of the C# mantra of 2005 ("Allocations are free! The garbage collector will save us all!"), but this was honestly never necessary. There should have been allocation-free ways of doing this.

From there, more heap layers join the party thanks to EndPoint itself being an abstract class. All an "endpoint" needs to do is combine an address and a port.

Strike 2: Invisible caching

/// <summary>
/// A lazily initialized cache of the result of calling <see cref="ToString"/>.
/// </summary>
private string? _toString;

/// <summary>
/// A lazily initialized cache of the <see cref="GetHashCode"/> value.
/// </summary>
private int _hashCode;

I mean, this doesn't bother me too much. I appreciate that the devs are keeping an eye on performance for the typical coder who just wants to Get Things Done™. Caching the hash code feels odd to me given that hash codes should always be relatively fast.

Regardless, I don't want to see this behavior native to a type. I'd rather have a Cached<T> wrapper for special cases.

Strike 3: Object-Oriented Nonsense

So, an IPAddress can be either an IPv4 address or an IPv6 address. I think I understand the desired convenience here: you don't really care what kind of address it is; you just want to store it one place. My problem is that there really isn't a situation where I need or want that.

If I open a socket and bind it to an IPv4 address, all addresses are 32 bits. If I bind it to an IPv6 address, all addresses are 128 bits. If I want to operate my socket in dual mode, the addresses are still always represented using 128 bits; an IPv4 address is simply represented in a special format.

If I know the exact sizes of my addresses before even opening the socket, why should I pay the price of all this garbage collector activity?

We can do better

Even setting aside all my gripes about allocations or performance, I just want a simpler API for working with UDP. In my next post, I'll cover the basics of networking in Jawbone.

C# Namespace Fever

Kelly Brown — Sun, 28 May 2023 16:39:32 +0000

Alright... I'm done chopping my code into microscopic namespaces. There's just no point. All it does is force a bunch of using statements all over the code (both in the project itself and in any project consuming it as a library).

The most common place I see this unhealthy obsession is in your typical ASP.NET app. Devs feel this compulsion to make a namespace for every kind of component.

namespace MyApp.Web.Controllers;
namespace MyApp.Web.Models;
namespace MyApp.Web.Services;
namespace MyApp.Web.Utils;

And this is a kind representation. I've even seen projects isolate Models.Request from Models.Response. Personally, I do not think this is necessary or helpful. In my more recent endeavors, I've settled on just using MyApp.Web and stopping there. It gets rid of a ton of useless using statements, and all the identifiers I need are now properly visible. I still find it acceptable to put the other whole layers into separate namespaces (MyApp.Domain, MyApp.Repository, etc.).

I fell prey to this in Jawbone. I think there is just something alluring about "being as cool as the .NET base classes". Hey, .NET has a namespace for its collections, so I should have a namespace for my collections! There's just one problem, though: .NET is huge, and my library is not. When you reach a certain size, it makes sense to gently categorize various components. Also, if code exists in distinct projects, having a different namespace makes sense.

So, I've taken the first steps to wrangling this mess. I am eliminating the Piranha.Jawbone.Tools and Piranha.Jawbone.Collections namespaces. In my mind, there are only two scenarios that justify a new namespace:

Strongly distinct domains: I am keeping namespaces such as Piranha.Jawbone.Sqlite and Piranha.Jawbone.Sdl. Those are highly cohesive domains that could arguably even be spun off into their own libraries if the need arose. They revolve around major native components and have clear delineation.
Isolation of extensions on common datatypes: if I want to introduce extensions for common types like string or DateTime or whatever, I don't want those extensions flooding devs' autocomplete. It's polite to let them opt in by using Piranha.Jawbone.Extensions. They should not pay that price just for using Piranah.Jawbone. Meanwhile, for new datatypes introduced by Jawbone itself, the extensions just sit in the Piranha.Jawbone namespace.

Already, I am enjoying the drastic reduction in using statements.

Jawbone: a new game dev library for .NET 5

Kelly Brown — Wed, 30 Dec 2020 04:07:15 +0000

(Obviously, time has passed, and I'm targeting .NET 7 now.)

I want to develop cross-platform games in modern C#. I have effectively retired from C++; I want to build my little game dev empire in the latest version of .NET. Frankly, I want nothing to do with .NET Framework or Mono anymore.

After a bunch of research and lots of disappointment (a whole blog post for another day), I have begun development of a new cross-platform game library. It is called Jawbone because it is a super important part of the piranha! (My company is "Obvious Piranha". That's the reference.) This middleware will simplify deploying to Windows, Mac, and Linux.

Features

This library provides access to SDL 2, OpenGL, OpenAL, STB (images, fonts, and Vorbis), and SQLite 3. In addition to raw access, there are abstractions that make working with these tools less painful.

The entire library makes use of nullable references. This will make it absolutely clear to consumers when null is an acceptable value. Jawbone will also make use of spans.

Continuing the tradition of projects such as SQLite and STB, all source code for Jawbone is public domain.

The native libraries are also all dynamically loaded, which makes it easier to pick and choose the components important to you. The native libraries are not automatically included. (Also, see this repo for building the public domain libraries for use with Jawbone.) This helps avoid inadvertently including a bunch of native binaries in a project that only wanted the C# code.

Jawbone makes heavy use of dependency injection.

There are many more individual features I hope to detail in future posts!

Roadmap

Before anyone becomes too excited, it is important to understand that the library is in a wildly immature state. The API contract is unstable and undocumented. I will most certainly post updates later on as the API matures. I have plans for three major phases.

Version 0

I am working on two games using this library. As a result, the design will shift every day. Building real games will help me harden the library's design and add needed features, but the library will be completely filled with sharp edges. It will be unfit for public consumption in this state as it will continue to be undocumented and incomplete as there will most certainly be features that other games need that my games happen to omit.

If you are a daring developer and choose to dive into the code at this stage, just know that you will run into things that don't make sense as there is still code leftover from when I began pursuing one design and shifted over to another. :)

Version 1

After completion of those two games, I will revisit the library and begin removing those sharp edges. I will refine the API into a user-friendly contract that makes sense and form pits of success. There will be documentation explaining proper use of the library as well more sample code. Completion of this phase is when I'll feel more comfortable spreading the news to places like Reddit and submitting binary builds to Nuget. :)

Version 2

Eventually, I want to reduce reliance on raw access to the various native libraries. Raw access will always be an option for power users, but I want to build an abstraction that is nice to use without sacrificing performance. I anticipate adding more robust access to OpenGL ES and Vulkan in this version as well.

Sneak Peek

To see the framework in action, I released a binary build of the sample application (currently Windows 64-bit only). Enjoy!

Less Magic, Please

Kelly Brown — Sat, 12 Dec 2020 19:29:41 +0000

I haven't posted in a while. I need to fix that. I have a lot of topics I wish to discuss in the near future. For now, here's a quick thought.

C# delegates are great, aren't they?

public string DoTheThing(Func<int, string> handler)
{
    return handler(_data);
}

See how I invoked that delegate? I simply treated it like a method: handler(_data). The issue is that delegates are objects, not methods. I've noticed in my own code that I've taken to writing this instead.

return handler.Invoke(_data);

I never consciously decided to invoke my delegates this way. It sort of just happened. Reflecting on this, I realized that being able to call delegates directly as if they are methods kinda hurts readability. I want my code to be just a bit more explicit. Let's look at another example.

if (flag == true)

Huh? What's going on here? Why bother with the == true part? Let's simplify that.

if (flag)

That's better. Except... C# is angry now.

Cannot implicitly convert type 'bool?' to 'bool'. An explicit conversion exists (are you missing a cast?)

Ooooooh. I get it. The == true part is a sneaky workaround to the fact that flag is a bool? or Nullable<bool>. In that case, I have a preferred way to handle this.

if (flag.GetValueOrDefault())

See, I don't like obscuring the fact that flag is a Nullable<bool>. I don't want to pretend that it's a flavored bool. The fact is Nullable<T> is its own type with its own rules. The code above makes it abundantly clear to any reader that we are dealing with a bool? and not a bool.

Agree or disagree. I like comments and discussion.

Dependency Injection in Game Dev

Kelly Brown — Mon, 17 Aug 2020 04:44:07 +0000

I am a huge fan of dependency injection. It is a central theme in ASP.NET Core, which I use extensively. In recent weeks, I became specifically interested in the dependency injection components themselves and learned how to use them outside of ASP.NET Core. It's remarkably simple to create your own ServiceCollection, populate it with services, and then build a ServiceProvider.

While that's all well and good, web development was the only domain I could think of where dependency injection would help. After all, server applications are long-running programs that need to juggle lots of components and connect everything together. I did not anticipate my interest in dependency injection and my interest in game dev colliding.

Here is an idea of the code powering top level systems in my engine.

using (var sqlite3 = LibraryMapper.Map<ISqlite3>(nativeDll, SqliteExtensions.ResolveName))
using (var stb = LibraryMapper.Map<IStb>(nativeDll, StbExtensions.ResolveName))
using (var al = OpenAlLoader.LoadOpenAl())
using (var audio = new AudioRenderer(al))
using (var sdl = SdlLoader.LoadSdl())
using (var windowManager = new WindowManager(sdl))
{
    // ...
}

So, not only do these systems need to initialize at the beginning, they need to be cleaned up at the end. From there, the systems begin progressively feeding in subsequent systems: AudioRenderer requires IOpenAl.

Originally, I was happy with this approach. I like how explicit and straightforward the code is. The order of events is clear. The object lifetimes are clear. What's not to like?

I started adding new subsystems. The first thing I did was begin replacing all calls to Console.WriteLine with calls to an instance of ILogger. This meant configuring the logging system up front and then injecting the logger into every system that is interested.

using var loggerProvider = new ConsoleLoggerProvider(new LocalConsoleLoggerOptions());
var logger = loggerProvider.CreateLogger("default");

using (var audio = new AudioRenderer(al, logger))
using (var windowManager = new WindowManager(sdl, logger))
{
    // ...
}

This one example wasn't that bad, but I began extracting internal subsystems and injecting them into the necessary main system because I needed access to those subsystems from other main systems. This outer method became somewhat of a nightmare to read and maintain. There were many local variables feeding into many constructors for various systems. I often had to return to the outer method, add stuff, reorder stuff, and come up with a dozen more variable names.

The nice thing about dependency injection frameworks is that they remove the burden of initializing and disposing services in the correct order. So, I decided to look into replacing all this with dependency injection. I was just going to run a little experiment. If I hated it, I could just revert it.

So, I made a service collection...

var services = new ServiceCollection();

...and filled it with stuff...

services
    .AddLogging(builder =>
    {
        builder.SetMinimumLevel(LogLevel.Trace);
        builder.AddConsole(options =>
        {
            options.IncludeScopes = true;
            options.DisableColors = false;
        });
    })
    .AddSqlite3(nativeDll)
    .AddStb(nativeDll)
    .AddOpenAl()
    .AddSdl2()
    .AddSingleton<AudioRenderer>()
    .AddSingleton<WindowManager>()
    .AddSingleton<Renderer>()
    .AddMetricRepository();

...and built a service provider.

// Make sure I didn't mess up.
var options = new ServiceProviderOptions
{
    ValidateScopes = true,
    ValidateOnBuild = true
};

using var serviceProvider = serviceCollection.BuildServiceProvider(options);

Now, I can pull services out, perform additional initialization, and start the game.

var windowManager = serviceProvider.GetRequiredService<WindowManager>();
windowManager.Run();

I gotta say: I'm very happy with the result. I was worried about losing the clarity of performing all these operations manually, but I much prefer the agility I'm afforded with this deferred initialization. Now, if I want to pull a subsystem into another system, it's as simple as adding it to the constructor.

public ImportantSystem(
    ILogger<ImportantSystem> logger,
    ItemSystem itemSystem,
    CraftingSystem craftingSystem)

I do not have to go back to the outer method, call the new constructor, pass the dependencies in, and possibly change the order of initialization. I also gain a new superpower: I can spawn objects and automatically inject all of its dependencies.

var widget = ActivatorUtilities.CreateInstance<Widget>(serviceProvider);

I highly doubt I am the first one to have this idea of using DI in a game engine, but it was satisfying to leverage a new bit of tech and simplify the code. Have you leveraged dependency injection outside of web development before? How did it go?

What else would I change in a C# rewrite?

Kelly Brown — Sat, 06 Jun 2020 22:38:57 +0000

Just reiterating: I love C#. I will use it forever. These are just thoughts about how I would approach a spinoff language.

Immutable Arrays

Increasingly, I wish T[] was immutable. It makes perfect sense that strings are immutable. As a result, there is cognitive dissonance in arrays being mutable. I wish I could concatenate two arrays the way I concatenate two strings.

.NET Core 2.1 introduced string.Create, which gives you temporary mutable access to the string's storage (Span<char>).

public static string Create<TState>(
    int length,
    TState state,
    SpanAction<char, TState> action)

A similar solution would work for arrays to avoid unnecessary copies. There could also be an overload that handles one element at a time.

public static T[] Create<T, TState>(
    int length,
    TState state,
    Func<TState, int, T> valueGenerator)

Eliminate Multicast Delegate

I can't remember the original reason for this decision. I believe it was done to support events in GUI frameworks.

It is easy to compose a custom multicast delegates from monocast delegates, but the reverse is simply not possible. We all pay for the cost of multicast delegates even though up to 100% of delegates in a project are monocast. That's an unfortunate fee for using delegates at all.

Fix `switch`

The switch statement was lifted straight from C/C++.

switch (input)
{
    case 1:
        DoTheThing();
        break;
    case 2:
    case 3:
        DoTheOtherThing();
        DoAnotherThing();
        break;
    default:
        DoDefaultThing();
        break;
}

Let's just adopt the if-statement scope rules.

switch (input)
{
    case (1)
        DoTheThing();

    case (2, 3)
    {
        DoTheOtherThing();
        DoAnotherThing();
    }

    default
        DoDefaultThing();
}

Sealed by Default

I don't think classes should be open to extension by default. Inheritance is a very particular superpower in the OOP landscape. I feel that a given class should have to invite inheritance rather than restrict inheritance. In other words, switch to opt-in rather than opt-out.

A class could invite inheritance by marking itself with either abstract (which requires inheritance anyway) or some other keyword like base.

public base class Widget

Drop Multidimensional Arrays

Treating an array as multidimensional data belongs to abstractions. Force developers to explicitly decide whether the grid is row-major, column-major, etc.

Drop `Equals`

The object virtual method Equals is a relic of the pre-generic era. If you want to know if two references are the same, call ReferenceEquals. Otherwise, it is up to the type itself to decide whether it has a "value" that can be equated to another. In such cases, I'd much rather work with IEquatable<T> anyway.

Drop `GetHashCode`

Following the section above, if two values are not meaningfully equatable, they're probably not meaningfully hashable either. Can two objects be hashable without being equatable? Maybe there should be an interface IKey<T> that extends IEquatable<T> and adds GetHashCode. Otherwise, an IHashable interface would do the trick. Maybe this could also serve as an opportunity to support hashes of different lengths: IHashable32, IHashable64, or others.

What would I change in a rewrite of C#?

Kelly Brown — Sat, 29 Feb 2020 18:36:02 +0000

Just to be clear, despite the title of this post, I love C# and will continue using it for as long as I can. In my opinion, it is the best designed general-purpose programming language available today. No, it's not perfect (as noted by the existence of this post), but I think C# does the least amount wrong, and it is a joy to work in.

The purpose of this post is simply to explore what kind of breaking changes would be worth making to the language if the opportunity were to present itself. I am a fan of programming language design. To me, this is just an exercise in design and evaluating the consequences of such design decisions. Not everything listed below is a breaking change; the assumption is that the change would deprecate a bunch of other APIs/behavior, which would be removed.

UTF-8 Everywhere

Historically, the choice to use UTF-16 for strings in Java and C# makes sense. I'm not mad about it, but today, the best choice is definitely UTF-8. It often has higher byte-density than UTF-16 (especially considering the programming language itself is English, as are popular protocols like HTTP). The new System.Text.Json APIs specifically provide ways to bypass conversion to and from string because the world outside of C# (mainly the web) has largely settled on UTF-8. The continued use of UTF-16 is a performance liability at this point.

Methods Outside Classes

Not much to say here. I want to be able to make free-floating methods. Stop insisting everything I do be inside a class or struct. OOP is great, but this requirement was an overreach. While we're at it, can I avoid indenting my entire file inside a namespace? Just let me do this up top:

namespace Acme;

Reader + Writer Paradigm

I really like the trend in recent .NET APIs to separate data types into explicit reader and writer types. Take a look at Channel<T>. The class is nothing more than a tuple of ChannelReader<T> and ChannelWriter<T>. This allows me to pass just the reader or just the writer into a method. It is much clearer what can and cannot happen. I want to take this concept and spread it just about every other corner of the language. I would love to have an ArrayReader<T> or a DictionaryWriter<TKey, TValue>. There were attempts at this in the early days with things like IReadOnlyCollection<T>, but they have two problems. First, the names are dishonest. An IReadOnlyDictionary<TKey, TValue> is not a read-only dictionary; it is simply read-only in a small context, but code elsewhere may very well be writing to it (possibly even concurrently). Second, it's often not that hard to simply cast to a read-write type. Obviously, that'd be a conventional no-no, but the point is that having a clean separation between Reader<T> and Writer<T> puts distance between the two modes.

With this design in place, APIs could be much clearer about their intent. There are many APIs, for example, that accept IDictionary<TKey, TValue>. Will it write to that dictionary? Probably not, but who can say for sure? I'd feel a lot better if I could send around IDictionaryReader<TKey, TValue>. In cases like ImmutableDictionary<TKey, TValue>, there simply wouldn't even be an IDictionaryWriter<TKey, Value> anywhere! You'd call transformation methods that return a new IDictionaryReader<TKey, TValue>.

The longer I code in C#, the more I wish arrays were immutable. I'm not fully sold on F# or functional programming in general, but if some kind of ArrayReader<T> were available (again, such that it couldn't just be cast to the writer version), I'd like that. So many legacy APIs (and even new ones) accept T[], which means I am at the API's mercy as to whether things inside will change. It feels like a subtle inconsistency that strings are immutable while arrays are not, but I get why it played out that way.

Note that I am aware of ReadOnlySpan<T> and ReadOnlyMemory<T>. They are great but do not fully accomplish what I described above.

Strict Nullability Rules

I love that C# 8 is tackling the problem of null references. I am adopting it into my current projects with great success, but this really should have been there from the beginning. The issue with C# 8 is that it is an opt-in mechanic that must remain compatible with code that did not opt-in. Again, I'm not criticizing the .NET team: adding this kind of feature this late into the ecosystem is a huge undertaking, but this post is about my dream version of C#. :)

I know people are a fan of things like Maybe<T> or Optional<T>. I've just always been a fan of leveraging null for that purpose. How great would it be if every single API used ? to denote nullability? Let's get rid of all that code that checks for null (unless, of course, you're working with the T? variant). I think I would even do things a little differently from C# 8. If I have a string? data variable, rather than trying to make the compiler super duper smart about if or when the variable is null, I think nullable types should have a one-way conditional cast to a non-nullable type. I'm not sure what the code would look like.

// Maybe something like this...?

void DoTheThing(string? data)
{
    if (data.TryGetValue(out string nonNull))
    {
        // nonNull would only be visible inside here.
    }

    // nonNull could not be used here.
}

I just think it would be simpler both for humans and compilers to understand where the transition happens and what contexts the non-null value exists in. The variable data would be more of a wrapper than a value (much like how they are for Nullable<T>) with fewer implicit casts to the non-null variant.

`defer` keyword

There is actually an open issue for this feature right now. I'm excited, provided it is done right. This keyword would effectively become a fancy finally block to help deal with objects that do not follow the IDisposable flow. So, today, you write code like this:

semaphore.Wait();

try
{
    // Do all the things.
}
finally
{
    semaphore.Release();
}

I don't like that the entire body of work is contained in that try block, which also results in another level of indentation. I would much rather write code like this.

semaphore.Wait();
defer semaphore.Release();

// Do all the things.

There are workarounds today, but I do not like that they largely entail extra objects and/or delegates.

semaphore.Wait();
using var sr = new SemaphoreReleaser(semaphore); // Custom type.

// Do all the things.

At least, in C# 8, it still avoids the new block/indentation thanks to the freestanding using.

To Be Continued

I'm sure I'll think of more ideas to share in the future. I can turn this into a series if there's interest. Feedback welcome! Have a great day!

Converting Dictionaries in System.Text.Json

Kelly Brown — Wed, 26 Feb 2020 05:02:30 +0000

Like many out there, I'm a long-time consumer of Newtonsoft.Json. It's simple and powerful, so it surprised me to learn that .NET Core 3 would be introducing its own JSON library. What surprised me more was learning that the man behind Newtonsoft.Json himself would be involved with designing this new library. This new library, System.Text.Json, was built with a specific purpose in mind: performance with JSON-related functionality in ASP.NET Core. It promises reduced allocations and higher throughput.

The design of this new library is somewhat off-putting for many. Right out of the gate, I wondered what happened to JObject, JArray, and JToken. They're just... gone... with no real replacement. The new library has JsonElement, but it's largely a read-only component. It is only created as part of a large parse operation; I can't just create JSON things and glue them together like I could with Newtonsoft.

I spent more time with it and slowly grew accustomed to its design philosophy. I no longer miss JObject because it was a thin wrapper over things I already have in .NET anyway. What is a JSON object really? It's just a Dictionary<string, object>.

var obj = new Dictionary<string, object>
{
    ["id"] = 43,
    ["title"] = "Western Union",
    ["isEnabled"] = true,
    ["tags"] = new string[] {"things", "stuff"}
};

var json = JsonSerializer.Serialize(obj);

Cool. Simple enough. I can also serialize custom objects with arbitrary properties to accomplish the same things. With this in mind, I kind of see the old Newtonsoft classes as crutches more than anything else. They offer feel-good measures for directly crafting JSON objects and arrays. (And I'm not faulting anyone for sticking with that paradigm. It's pretty easy to read.)

However, I was caught off guard when I went to deserialize a dictionary.

var raw = "{\"49fc2162-744a-4a42-b685-ea1e30ce2a2f\": 99}";
var d = JsonSerializer.Deserialize<Dictionary<Guid, int>>(raw);

This code here throws a NotSupportedException. It points out that having a non-string for the TKey is simply unacceptable. This... is a problem. I work with Guid extensively (at home and at work) and often use it as a dictionary key. I'm generally not OK with having to convert JSON to a string dictionary first and then to a non-string dictionary as a second step. It seems I am not alone in experiencing frustration at this.

However, as much as I love to sit and complain about others' hard work, I decided to just do something about it. The documentation for creating your own JSON converters is pretty good. So, I set out to make my own, and I am pleased with the result so far. Now I can do this:

var options = new JsonSerializerOptions();
options.Converters.Add(DictionaryJsonConverterFactory.Default);

var raw = "{\"49fc2162-744a-4a42-b685-ea1e30ce2a2f\": 99}";
var d = JsonSerializer.Deserialize<Dictionary<Guid, int>>(raw, options);

Back in business! If you are interested in my implementation, the code is all here with a small explanation here. It's public domain. Knock yourself out.

Why are these the gitignore rules for VS Code?

Kelly Brown — Sun, 15 Dec 2019 18:57:07 +0000

If you search for good .gitignore rules for Visual Studio Code, you often come across these.

##
## Visual Studio Code
##
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json

I tried using them for a while, but they just lead to problems. It is the equivalent of committing the .user files in other .NET projects. The file launch.json, for example, contains my own command-line arguments, environment variables, etc. Why would I want those committed? Why would I want changes from other developers overriding mine all the time? Who decided this was a good idea?

It's particularly annoying because dotnet new gitignore imports these questionable rules. I just block the entire folder now.

.vscode/

If I ever clone my repo from scratch, VS Code offers to whip up a new set of files for me anyway. Are there other languages/environments where committing these files is extremely beneficial?

How to Conditionally Target WinExe

Kelly Brown — Thu, 28 Nov 2019 22:35:59 +0000

When creating cross-platform games in .NET Core, you'll typically use the dotnet new console template for the project. When creating a console app, you end up with a project file like this:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
  </PropertyGroup>

</Project>

Running this application under Windows results in a console window showing up even if the bulk of the application runs inside a graphical window (created by SDL or some other middleware). The console window is fine for development but no good when it comes time to ship the application. Unlike macOS or Linux, applications in Windows have a flag embedded into the binary itself dictating whether it should spawn a console window.

Before .NET Core 3.0 was released, I could only find one real option to deal with this: the tool editbin.exe, which ships with Visual Studio (not VS Code).

editbin.exe /subsystem:windows yourapp.exe

And this works, but I hate that my build environment requires editbin.exe be installed and added to my PATH. It's just more stuff that can go wrong; I wanna keep things simple.

Thankfully, .NET Core 3.0 introduced a new way to solve this problem: simply change your OutputType to WinExe in your .csproj file! I much prefer this solution because it brings me back to only needing the .NET Core SDK and nothing else. Complexity is the enemy, and I will slay it whenever I can.

However, permanently changing my OutputType complicates things: I want that console window almost all the time during development. Not only does it let me see my console debug output, it gives me another handy way to close the application in case it hangs or something: I can just close the console window itself. So, ideally, I want the OutputType to be Exe except when publishing my application.

So, initially, I created my script publish.cmd to contain this:

dotnet publish -c Release -r win-x64 -p:CopyOutputSymbolsToPublishDirectory=false -p:OutputType=WinExe

And this works... but not really. The problem is that the moment you introduce a dependency to your project, this path fails. Let's say you are building a shared library the same time you are building your desktop application.

<ItemGroup>
  <ProjectReference Include="..\Kelly.Shared\Kelly.Shared.csproj" />
</ItemGroup>

You will suddenly run into this error:

CSC : error CS5001: Program does not contain a static 'Main' method suitable for an entry point [C:\Users\kelly\Documents\Kelly.Shared\Kelly.Shared.csproj]

What is going on here? It turns out that when you specify -p:OutputType=WinExe, it happens globally. The property is propagated to all projects being built. So, your shared library is no longer happy to just be a library; it wants to be an executable! And that just can't happen because, as the error points out, it has no Main method.

So, back to the drawing board. How can I clearly indicate that I want only Kelly.Client to be WinExe? Well, I solved it by introducing a property into my project that no other project cares about.

<PropertyGroup>
  <KellyClientOutputType>Exe</KellyClientOutputType>
  <OutputType>$(KellyClientOutputType)</OutputType>
  <TargetFramework>netcoreapp3.0</TargetFramework>
</PropertyGroup>

Now my project looks to the property KellyClientOutputType to know what the output type is. This means I can alter my script to change this other property, and it will not affect any dependencies.

rmdir /s /q bin\Release
rmdir /s /q obj\Release
dotnet publish -c Release -r win-x64 -p:CopyOutputSymbolsToPublishDirectory=false -p:KellyClientOutputType=WinExe

Voila. Now I can just run this script whenever I wish to make a clean build that I want to share with outside parties.

Building a Better Library Loader in C# - Part 3

Kelly Brown — Thu, 18 Jul 2019 03:50:27 +0000

In part 2, I went over the creation of a type at run-time by way of System.Reflection.Emit. We are able to define an interface we want to use, and our handy dandy LoadLibrary<T> method will take care of all the details for us. That, by itself, is pretty magical.

But we're not done. We need a few quality-of-life improvements.

Returning Strings

Let's go back to our SQLite interface.

interface ISqlite3
{
    int Open(string file, out IntPtr database);
    int Close(IntPtr database);
}

See how that Open method takes a string for one of its parameters? It really shouldn't work because System.String is a radically different data structure from the const char* that sqlite3_open expects. The first problem is that System.Char is 2 bytes, and the char in C is 1 byte. The second problem is that strings/arrays in .NET embed their length, and C strings use null-termination.

Thankfully, .NET foresaw this and took care of the issue for us. So, this methods works as you would expect! You can simply call sqlite3.Open("stuff.sqlite", out var db); without difficulty because your string is automatically converted to a null-terminated C string that your C library can actually handle.

However, what if we wanted a method that returns a string? Let's look at sqlite3_errmsg.

string Errmsg(IntPtr database);

I have bad news for you. If you call this, your program will crash. If it doesn't crash, it'll behave badly regardless. The nice type marshaling we had for values going in apparently does not apply to values coming out. So, how do we fix this? Well, since the C function returns a pointer (const char*), we need to capture it as a pointer.

IntPtr Errmsg(IntPtr database);

Great. How do I get a string out of that? Well, if you're using .NET Core, the answer is simple.

public static string GetString(IntPtr cString)
{
    return cString == IntPtr.Zero ? null : Marshal.PtrToStringUTF8(cString);
}

If you're not using .NET Core... shame on you. Start using .NET Core immediately! If you absolutely cannot use .NET Core (or a sufficiently high enough version of .NET Standard), you are going to have to parse the string yourself.

public static string GetStringTheHardWay(IntPtr cString)
{
    if (cString == IntPtr.Zero)
        return null;

    var bytes = new List<byte>();
    int n = 0;

    while (true)
    {
        byte b = Marshal.ReadByte(cString, n++);

        if (b == 0)
            break;

        bytes.Add(b);
    }

    return Encoding.UTF8.GetString(bytes.ToArray());
}

OK, so, now we have a way to read strings returned from C functions, but this is still too much work. Now, every time I want a string from my method, I have to call GetString(returnedValue). That's no good! I want to be lazier than that! So, let's revisit our code generator.

ILGenerator generator = methodBuilder.GetILGenerator();

for (int i = 0; i < parameters.Length; ++i)
    generator.Emit(OpCodes.Ldarg, i + 1); // Arg 0 is 'this'. Don't need that one.

if (Environment.Is64BitProcess)
    generator.Emit(OpCodes.Ldc_I8, ptr.ToInt64());
else
    generator.Emit(OpCodes.Ldc_I4, ptr.ToInt32());

generator.EmitCalli(
    OpCodes.Calli,
    CallingConvention.Cdecl,
    method.ReturnType,
    parameterTypes);

generator.Emit(OpCodes.Ret);

The Calli instruction places the C function's return value onto the stack, and the subsequent Ret instruction forwards it onto the caller. There is an opportunity here: we know, ahead of time, what the return type is. Why not just handle the conversion right here? If we know that the method on the interface wants to return string, we can take care of returning IntPtr internally and converting it before returning it.

// This should be stored outside the loop iterating over the interface methods.
MethodInfo getStringMethod = typeof(WhateverClassHasTheMethod).GetMethod(nameof(GetString));

ILGenerator generator = methodBuilder.GetILGenerator();

for (int i = 0; i < parameters.Length; ++i)
    generator.Emit(OpCodes.Ldarg, i + 1); // Arg 0 is 'this'. Don't need that one.

if (Environment.Is64BitProcess)
    generator.Emit(OpCodes.Ldc_I8, ptr.ToInt64());
else
    generator.Emit(OpCodes.Ldc_I4, ptr.ToInt32());

bool returnsString = method.ReturnType == typeof(string);

generator.EmitCalli(
    OpCodes.Calli,
    CallingConvention.Cdecl,
    returnsString ? typeof(IntPtr) : method.ReturnType,
    parameterTypes);

if (returnsString)
    generator.Emit(OpCodes.Call, getStringMethod);

generator.Emit(OpCodes.Ret);

Do you see what we did? The return type sent off to EmitCalli now has special treatment. If we ultimately want a string to come back, we capture an IntPtr first. Since that IntPtr value is on top of the stack after the Calli instruction, it is in the right position to serve as an argument for the call to GetString. How convenient! The call to GetString will place the string result onto the stack, and then our Ret instruction will return it to the caller.

Voila. Methods that return string work fine now.

Library Cleanup

Remember way back in part 1 when we talked about loading and unloading libraries? Our fancy-shmancy LoadLibrary<T> happily loads up the library but leaves us without a way to unload the library. That may not matter to many developers. It's quite normal for games to load up libraries, use them for the entire lifetime of the application, and then skip unloading them at the end. Regardless, I think it's best we at least offer up some way to unload these libraries as there are applications out there that need to load and unload libraries over time.

There are several approaches we can take here. I suggest leveraging IDisposable as it is idiomatic and brings several benefits along with it. The top benefit (in my mind) is the ability to put your library into a using block.

using (ISqlite3 sqlite3 = LoadLibrary<ISqlite3>("custom_sqlite3.dll", GetSqliteFunctionName))
{
    sqlite3.Open("my_data.sqlite", out IntPtr database);
    // ...
}

Another perk is that we can enforce this in the LoadLibrary<T> definition itself. Forgetting to mark your interface as IDisposable will result in a compiler error.

public interface ISqlite3 : IDisposable
{
    int Open(string file, out IntPtr database);
    int Close(IntPtr database);
}

public static T LoadLibrary<T>(
    string library,
    Func<string, string> methodNameToFunctionName)
    where T : class, IDisposable
{
    // ...
}

So, let's make a helper method to implement the Dispose method.

private static void CreateDisposeMethod(
    TypeBuilder typeBuilder,
    IntPtr libraryHandle)
{
    MethodBuilder methodBuilder= typeBuilder.DefineMethod(
        nameof(IDisposable.Dispose),
        MyMethodAttributes);
    typeBuilder.DefineMethodOverride(
        methodBuilder,
        typeof(IDisposable).GetMethod(nameof(IDisposable.Dispose)));

    Type ptrType = Environment.Is64BitProcess ? typeof(long) : typeof(int);
    ConstructorInfo intPtrConstructor = typeof(IntPtr).GetConstructor(new Type[] {ptrType});
    MethodInfo closeMethod = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) ?
        typeof(WinLoader).GetMethod(nameof(WinLoader.FreeLibrary)) :
        typeof(UnixLoader).GetMethod(nameof(UnixLoader.Close));

    ILGenerator generator = methodBuilder.GetILGenerator();

    if (Environment.Is64BitProcess)
        generator.Emit(OpCodes.Ldc_I8, libraryHandle.ToInt64());
    else
        generator.Emit(OpCodes.Ldc_I4, libraryHandle.ToInt32());

    generator.Emit(OpCodes.Newobj, intPtrConstructor);
    generator.Emit(OpCodes.Call, closeMethod);
    generator.Emit(OpCodes.Pop); // Toss the returned int.
    generator.Emit(OpCodes.Ret);
}

As you can see, this method has to construct the IntPtr instance because there is no way to embed a constant IntPtr into the code. As I mentioned before, I recommend just using SharpLab in helping to determine what IL you want to emit. I'm no IL expert; I just write C# code for what I want to accomplish and study the resulting IL.

Just remember to call CreateDisposeMethod somewhere in your LoadLibrary<T> method, and you're gold. In part 4, we'll do more upgrades specific to game dev. :)