Designing Around F# Interop

Last year I wrote about Advanced F# Interop techniques. It was based on tackling the least ideal situation of language interop: A C# (or VB) library that was never designed with F# in mind. The techniques used were straight up abuse of the language, but were able to warp incredibly non-FP code into FP code.

I still want to cover these points, but I don't want C# and VB programmers thinking it's acceptable to never consider interop. There's techniques which make it easier to design C# and VB libraries while also making it easier to create the F# bindings. And yes, there's techniques for doing this the other way around as well, which I'll cover in time.

For almost two years now I've been developing and maintaining a set of libraries where cross language compatibility is a hard design requirement. Obviously, I want this to be effective while not requiring a ton of effort. This article covers my findings, and integrated contributions of others towards that goal. The hope is that this will help library authors support the entire .NET ecosystem with a better overall feel, because we're better together.

The Past

A quick primer on what was done in the past, if you hadn't read my previous article. I'll be carving it up, simplifying it immensely by the end of this.

I turned this:

let stack = DoubleStack()
let mutable result = ref 0.0
stack.Push(3.0)
stack.Push(5.0)
stack.Multiply(result)

into this:

let stack = 3.0 |=> 5.0 |> mul |=> 8.0 |> sub
Assert.Equal(7.0, peek stack)

mostly through this:

type Pipeline =
    static member Pipe(left:DoubleStack, right:float) =
        left.Push(right)
        left
    static member Pipe(left:float, right:float) =
        let result = DoubleStack()
        result.Push(left)
        result.Push(right)
        result

let inline private pipe< ^t, ^a when (^t or ^a) : (static member Pipe : ^a * float -> DoubleStack)> left right =
    ((^t or ^a) : (static member Pipe : ^a * float -> DoubleStack)(left, right))

let inline ( |=> )(left:^a)(right:float) = pipe<Pipeline, ^a> left right

I don't want to go deep into explaining because we're going to remove much of this, but:

|=> doesn't need to be an operator, I was just trying to be creative. Any F# function will work, and currying is recommended but not required. This tells the compiler it needs to resolve types at the callsite. pipe is what sets up the binding. It looks for, and then calls, a static Pipe() on either ^t or ^a. In |=> we've said that ^t is Pipeline, which is where Pipe() is declared. You can't specify only ^t, as multiple type parameters are required to "confuse" the compiler.

The end result is when the public function, |=> in this case, is compiled, the compiler attempts to resolve a Pipe() with a signature that matches the types at the callsite.

In a normal C# binding situation Pipe() would be calling C# methods, or pipe would be a member binding, not a static member binding. But as I said, we're going to design around simplifying this entire thing.

Traits

I've wrote previously on how C# does support traits and even added it as far back as C# 3.0. Borrowing a key definition from that article, we have:

Let's make sure we're all on the same page about what a trait is. This will also be useful in explaining why this approach works.
A trait is a specific feature of a type. A method it supports. A property. Whatever. It's just saying: "I have this thing".

On the C# side this is actually a little less than idea. It's limiting in that unlike a first-class trait system, there's no way in C# to add traits to existing types. Extension methods allow for adding a method with the same name and signature as it would have if there was first-class traits, but this is less than ideal.

F#'s functions have one of the more powerful and sophisticated generic constraint systems I've seen, only being bested by Ada. Because of this sophistication it's very easy to define trait-based functions in F#. In fact, you've certainly done this normally when writing F# functions, even without explicit constraints.

So what we're left with is using a trait-pattern in C# allows for a high degree of consistency in name and signature, which F# bindings need. But also inconsistency in whether those methods are instance or static members. That's a problem.

Null tollerance

Admittedly, I stumbled upon this completely unrelated to anything regarding F#, but quickly realized its application here as well. I try to avoid exceptions. They have their role, and I'll certainly use them before I start returning Int32 as an error code. But they are heavy. NullArgumentException and NullDereferenceException are particularly annoying ones for many. I happened to have many instances where a well defined action could be taken if the caller was null. But if instance methods always have an implicit caller, how do we check if that's null? This was always something that bugged me about C#, having also programmed extensively in Ada. Conceptually, I knew how to deal with this; I knew it shouldn't be anywhere near as big of a problem.

I worked out how, and wrote about it here. If you haven't read that entire article, the short version is roughly: if you want to know how many items are in a bag, and you don't even have a bag, how many items do you have? You don't get a NullDereferenceException irl, you have no items. So let's code for that by adding null-tollerance into extension methods, then calling the non-public instance methods. Cumbersome for many? Maybe. I figured it was most applicable for library authors, and only when well defined graceful action could be taken for a null caller. But it had unintentional side effects.

I could now provide default interface implementations and have them show up across every type with that trait, callable like normal instance methods.

And...

I just pushed all methods into static classes. F# functions had a fully consistent constraint to bind to.

Refined F# Bindings / Putting It All Together

Unlike before, I'm lifting these examples directly from the libraries I've mentioned, so you can see a real life applied case of this.

public interface IAddable<TElement> {
    void Add(TElement element);

    void Add(params TElement[] elements) {
        if (elements is null) {
            return;
        }
        foreach (TElement element in elements) {
            Add(element);
        }
    }

    void Add(Memory<TElement> elements) => Add(elements.Span);

    void Add(ReadOnlyMemory<TElement> elements) => Add(elements.Span);

    void Add(Span<TElement> elements) => Add((ReadOnlySpan<TElement>)elements);

    void Add(ReadOnlySpan<TElement> elements) {
        foreach (TElement element in elements) {
            Add(element);
        }
    }

    void Add<TEnumerator>(IEnumerable<TElement, TEnumerator> elements) where TEnumerator : IEnumerator<TElement> {
        if (elements is null) {
            return;
        }
        foreach (TElement element in elements) {
            Add(element);
        }
    }
}

Here we have a straightforward but featureful "trait". All an implementer needs to provide is an implementation of Add(TElement), and they get the {,ReadOnly}{Memory,Span}<TElement> and IEnumerable<TElement, TEnumerator> variants for free. Why is IEnumerable<> different? You can ignore that, it's a library specific thing; you want to use the one in System.Collections.Generic. Like mentioned before, on its own this really isn't useful, since you need to call these off the interface, which forces a very particular style that is problematic with such "anemic" interfaces.

public static partial class TraitExtensions {
    public static void Add<TElement>(this IAddable<TElement> collection, TElement element) {
        if (collection is null) {
            return;
        }
        collection.Add(element);
    }

    public static void Add<TElement>(this IAddable<TElement> collection, params TElement[] elements) {
        if (collection is null) {
            return;
        }
        collection.Add(elements);
    }

    public static void Add<TElement>(this IAddable<TElement> collection, Memory<TElement> elements) {
        if (collection is null) {
            return;
        }
        collection.Add(elements);
    }

    // And so on...
}

Whether Add() should do nothing with a null collection is debatable, of course. My thinking was similar to writing to /dev/null versus an actual file.

This is where four of the important things I had mentioned took place. We've now added graceful handling of null callers. We've now allowed the default interface implementations to be called off the implementing type, not just the interface. We've greatly reduced the amount of code in large code bases by providing single implementations of many things. And, if we need to add And() or whatever other method to a preexisting type, we can do that inside of this static class and have everything consistent for the F# side of things.

Lastly, the F# binding side is just this now:

module internal Bindings =
    let inline Add< ^t, ^a, ^b when (^t or ^a) : (static member Add : ^a * ^b -> unit)> collection elements = ((^t or ^a) : (static member Add : ^a * ^b -> unit)(collection, elements))

[<AutoOpen>]
module Functions =
    let inline add (elements) (collection) =
        Add<TraitExtensions, _, _> collection elements
        collection

This works like before, but I'm going to summarize again for those unfamiliar with the previous approach, and to be extra clear about how this interacts with the newer design.

You call add, which, due to being inline has its types resolved at the callsite. It, in turn, calls Add, which specifies TraitExtensions as a place to look for members, and says the other parameter types can be anything. Then, because an F# function is generally expected to return something other than unit, it returns the collection which can be further pipelined.

Add sets up the binding just like before. It says to look for a static member on ^t or ^a called Add, with the signature ^a * ^b -> unit. Because ^t is always TraitExtensions, then what is being said is specifically: Look on TraitExtensions and the type of collection for the static member: Add : ^a * ^b -> unit.

We now have a function, add (elements) (collection) which will work for any collection of IAddable<TElement>, and allow adding any of the overloads of elements defined in IAddable<TElement>.

No seriously. Here's examples lifted directly out of my F# unit tests.

let array = DynamicArray<Int32>(8n)

Assert.Equal(0n, array.Length)

array

|> add 1

|> ignore

Assert.Equal(1n, array.Length)

Assert.Equal(1, array.[0n])

array

|> add [| 2; 3; 4 |]

|> ignore

Assert.Equal(4n, array.Length)

Assert.Equal(1, array.[0n])

Assert.Equal(2, array.[1n])

Assert.Equal(3, array.[2n])

Assert.Equal(4, array.[3n])

Summary

Part of what I find brilliant about this solution is not only are the F# bindings easier to write, they're also easier to maintain. Previously, I struggled with ensuring there was strict 1:1 parity between them, but now that just happens every time I add new overload. While this approach would add work to very small libraries, it reduced the amount of code by roughly 11% (I'm not done some changes and don't have a precise number) without removing a single feature.

Would this be easier the other way around? Implement in F# and provide C#/VB bindings? I'm not sure, but it's something I'm very interested in and need to explore. Certain features F# was lacking kept me from trying this before, but with F# 5.0 I believe everything I need is now present. This, and some other things will be a topic for the future.

Now, let me be clear: this still isn't the most ideal situation. What I would love is for the C# team to provide ways to generate the same metadata F# uses. Things like being able to decare a C# static method has a curried form that F# can use without any bindings necessary. But with the current state of things, if you have to implement in C# and want to provide good functional and pipeline focused experiences for F# consumers, this seems to be one of the better ways to go about it.

Going Further

Just a few days before this article, I came across Gustavo Leon's experiment with Variadic Functions. I've come to expect flack from some of the more purist/fundamentalist members of the F# community, so I don't mind talking about this and even experimenting with whether it's useful in this context. Is it a "haskellism"? Yes. Is it potentially useful in this context? Yes. Should it be done in a pure F# context? Probably not. I'll report back later on whether F#-first/C#-bindings or variadic functions have uses in the context of multi-language libraries.