If we fix a bug by making a C# program more type safe, how do we test this?
Say for example you arrange for invalid code to not compile anymore?
The case
Active Logic uses a modified Kleene logic to implement Behavior Trees using the short-circuiting operators &&
and ||
. In addition of a three-valued type, we have restricted types to represent ahead-of-runtime knowledge. Anyway this is handy because it avoids logical errors (we love uncertainty, in moderation).
So here's an example of an operation that we're disallowing:
impending x;
status y;
var z = x && y;
impending represents a task that's either running or failing; whereas status (typical of BT) is either running, failing or done. In the above example, y (which normally is an expression, not a pre-assigned value) would never evaluate.
So this is something we can enforce at compile time, and the question then becomes: how do you put this under test.
Initially, our test looked like this:
[Test] public void Impending_AND_Status(){
impending x = impending_fail;
status y = done;
var z = x && y; // CS217
}
With our implementation this raises CS217 at compile-time, invalidating the (impending) && (status)
combination.
Expected behavior; also, who broke my test suite?
The dynamic keyword lets you disable compile checks:
[Test] public void Impending_AND_Status(){
dynamic x = impending.fail, y = status.done;
var z = x && y;
Print($"{x} && {y} => {z}");
}
...and you'd expect a runtime error to assert against. However (in our case) things get a little more complicated because the above DOES run and produce an output.
The compiler sees an invalid logical AND, but the runtime takes a more incremental approach:
1) Check the left hand for "falsehood". If the left hand is false return the left hand.
2) Type-check the right hand. If there is an applicable &
operation of the form lh & rh, evaluate the right hand.
3) Invoke the (user defined) &
operator on the resulting operands.
With this in mind, I then rewrote the false
operator for the impending type:
public static bool operator false(impending s)
=> throw new InvOp("Cannot test falsehood (impending)");
NOTE: In C#, true
and false
work in pairs. You can't implement one and not another. pending
should just never allow &&
. But this, ultimately, is not something we can enforce at compile time.
And our test then looks like this:
[Test] public void Impending_AND_Status(){
dynamic x = impending_fail, y = done;
Assert.Throws<InvOp> ( () => z = x && y );
}
Downside of course, the compiler and runtime aren't checking the same thing. Strictly we'd have to forgo NUnit, and write a script that:
1) Runs a build on invalid code
2) Verifies every error issued by the compiler
This approach, however, is heavy-handed, so I decided to go ahead with runtime checking and here's the final version of the test, as it will look in the next commit:
// o(x, y) // assert x equals y
// s(x) // new status from an int
// i(x) // new 'impending' value from int
[Test] public void Impending_x_Status([Range(-1, 0)] int x,
[Range(-1, 1)] int y){
AND_CS217_InvOp(i(x), s(y));
o( i(x) || s(y), s(x) || s(y) );
}
AND_CS217_InvOp
encapsulates the NUnit call (which is a bit long winded when you have (7 x 7 x 2) cases to cover). The compile error is still no more than a label, still useful for documenting our APIs.
Cooler talk
With runtime testing, compile-safe techniques are in a blind spot. This doesn't sit too well with me because testing is supposed to guide you towards better code. Is compile time safety not a good thing, then?
When you fix a bug or add a feature by tweaking type safety, if you don't have a test you also can't ensure your bug fix/feature won't be removed accidentally. You can roll your own tools, and dynamic
is a poor man's version of doing just that. Oh, and it helps understanding how the C# runtime even works.
Photo by Ali Sarvari on Unsplash
Top comments (0)