9 "Black Magics" and "Tricks" in C

We know that C# is an extremely advanced language due to its "syntactic sugar". These "sugar" sometimes are too convenient that some people might think they are written hard in the C# compiler, irrational—somewhat like "black magic".

So, let's see if these advanced features in C# are hard-written by the compiler ("black magic"), or extendable ("tricks") "duck typing".

I'll list a directory first. You can try to make a judgment based on this directory, whether it is "black magic" (hard-written by compiler) or "duck typing" (custom "tricks"):

1. LINQ operations, with IEnumerable<T> type;
2. async/await, with Task/ValueTask types;
3. Expression trees, with Expression<T> type;
4. Interpolated strings, with FormattableString type;
5. yield return, with IEnumerable<T> type;
6. foreach loop, with IEnumerable<T> type;
7. using keyword, with IDisposable interface;
8. T?, with Nullable<T> type;
9. Generic operations of Index/Range for any type.

1. LINQ operations, with IEnumerable<T> type

Not "black magic", but "duck typing".

LINQ is a new feature released with C# 3.0, which can operate on data conveniently. It's been 12 years now. Although some functions need to be enhanced, it's still much more convenient compared to other languages.

As I mentioned in my last blog, LINQ does not necessarily have to be based on IEnumerable<T>. Just define a type and implement the required LINQ expressions. The LINQ's select keyword will call the .Select method. You can use the following "tricks" to implement the effect of "transferring flowers and trees":

void Main()
{
    var query = 
        from i in new F()
        select 3;

    Console.WriteLine(string.Join(",", query)); // 0,1,2,3,4
}

class F
{
    public IEnumerable<int> Select<R>(Func<int, R> t)
    {
        for (var i = 0; i < 5; ++i)
        {
            yield return i;
        }
    }
}

2. async/await, with Task/ValueTask types

Not "black magic", but "duck typing".

async/await was released with C# 5.0, which can do asynchronous programming very conveniently. Its essence is a state machine.

The essence of async/await is to find a method named GetAwaiter() under a type. This method must return a class inheriting from INotifyCompletion or ICriticalNotifyCompletion, which also needs to implement the GetResult() method and IsComplete property.

This is explained in the C# language specification. Calling await t will execute in the following order:

1. First call the t.GetAwaiter() method to get the waiter a;
2. Call a.IsCompleted to get a boolean type b;
3. If b=true, then immediately execute a.GetResult(), to get the running result;
4. If b=false, then depending on the situation:
   1. If a does not implement ICriticalNotifyCompletion, execute (a as INotifyCompletion).OnCompleted(action)
   2. If a has implemented ICriticalNotifyCompletion, execute (a as ICriticalNotifyCompletion).OnCompleted(action)
   3. The execution is suspended afterward, and returns to the state machine after OnCompleted is completed;

If you are interested, you can visit the specific specification explanation on Github: https://github.com/dotnet/csharplang/blob/master/spec/expressions.md#runtime-evaluation-of-await-expressions

Normally Task.Delay() is based on a thread pool timer. You can implement a single-threaded TaskEx.Delay() with the following "tricks":

static Action Tick = null;

void Main()
{
    Start();
    while (true)
    {
        if (Tick != null) Tick();
        Thread.Sleep(1);
    }
}

async void Start()
{
    Console.WriteLine("Start execution");
    for (int i = 1; i <= 4; ++i)
    {
        Console.WriteLine($"The {i}th time, time: {DateTime.Now.ToString("HH:mm:ss")} - Thread number: {Thread.CurrentThread.ManagedThreadId}");
        await TaskEx.Delay(1000);
    }
    Console.WriteLine("Execution completed");
}

class TaskEx
{
    public static MyDelay Delay(int ms) => new MyDelay(ms);
}

class MyDelay : INotifyCompletion
{
    private readonly double _start;
    private readonly int _ms;

    public MyDelay(int ms)
    {
        _start = Util.ElapsedTime.TotalMilliseconds;
        _ms = ms;
    }

    internal MyDelay GetAwaiter() => this;

    public void OnCompleted(Action continuation)
    {
        Tick += Check;

        void Check()
        {
            if (Util.ElapsedTime.TotalMilliseconds - _start > _ms)
            {
                continuation();
                Tick -= Check;
            }
        }
    }

    public void GetResult() {}

    public bool IsCompleted => false;
}

The running effect is as follows:

Start execution
The 1th time, time: 17:38:03 - Thread number: 1
The 2th time, time: 17:38:04 - Thread number: 1
The 3th time, time: 17:38:05 - Thread number: 1
The 4th time, time: 17:38:06 - Thread number: 1
Execution completed

Note that it is not necessary to use TaskCompletionSource<T> to create a user-defined async/await.

3. Expression trees, with Expression<T> type

It's "black magic". There is no "room for operation", only when the type is Expression<T>, it will be created as expression trees.

Expression trees were released with C# 3.0 along with LINQ, which is a far-sighted "black magic".

For the following code:

Expression<Func<int>> g3 = () => 3;

It will be translated by the compiler as:

Expression<Func<int>> g3 = Expression.Lambda<Func<int>>(
    Expression.Constant(3, typeof(int)), 
    Array.Empty<ParameterExpression>());

4. Interpolated strings, with FormattableString type

It's "black magic". There is no "room for operation".

Interpolated strings were released with C# 6.0. Many languages have provided similar features before that.

Only when the type is FormattableString, it will produce different compilation results. As in the following code:

FormattableString x1 = $"Hello {42}";
string x2 = $"Hello {42}";

The compiler generates the following results:

FormattableString x1 = FormattableStringFactory.Create("Hello {0}", 42);
string x2 = string.Format("Hello {0}", 42);

Please note that it essentially calls FormattableStringFactory.Create to create a type.

5. yield return, with IEnumerable<T> type;

It's "black magic", but there are additional notes.

yield return can be used not only for IEnumerable<T>, but also for IEnumerable, IEnumerator<T>, IEnumerator.

Therefore, if you want to simulate the behavior of generator<T> in C++/Java with C#, it will be simpler:

var seq = GetNumbers();
seq.MoveNext();
Console.WriteLine(seq.Current); // 0
seq.MoveNext();
Console.WriteLine(seq.Current); // 1
seq.MoveNext();
Console.WriteLine(seq.Current); // 2
seq.MoveNext();
Console.WriteLine(seq.Current); // 3
seq.MoveNext();
Console.WriteLine(seq.Current); // 4

IEnumerator<int> GetNumbers()
{
    for (var i = 0; i < 5; ++i)
        yield return i;
}

yield return—"iterator" was released with C# 2.0.

6. foreach loop, with IEnumerable<T> type

It's "duck typing". There is a "room for operation".

foreach does not necessarily have to be used with the IEnumerable<T> type, as long as the object has a GetEnumerator() method:

void Main()
{
    foreach (var i in new F())
    {
        Console.Write(i + ", "); // 1, 2, 3, 4, 5, 
    }
}

class F
{
    public IEnumerator<int> GetEnumerator()
    {
        for (var i = 0; i < 5; ++i)
        {
            yield return i;
        }
    }
}

Besides, if the object has implemented GetAsyncEnumerator(), await foreach can also be used for asynchronous looping:

async Task Main()
{
    await foreach (var i in new F())
    {
        Console.Write(i + ", "); // 1, 2, 3, 4, 5, 
    }
}

class F
{
    public async IAsyncEnumerator<int> GetAsyncEnumerator()
    {
        for (var i = 0; i < 5; ++i)
        {
            await Task.Delay(1);
            yield return i;
        }
    }
}

await foreach was released with C# 8.0 along with asynchronous stream. For details, you can refer to my previous article "Code demonstration of new functions in each version of C#".

7. using keyword, with IDisposable interface

Yes, and no.

For reference types and normal value types with using keyword, it must be based on IDisposable interface.

But ref struct and IAsyncDisposable tell a different story. Since ref struct does not allow arbitrary movement, and reference types—managed heap, will allow memory movement, so ref struct is not allowed to have any relationship with reference types. This relationship includes inheriting interface—because interface is also a reference type.

But the need to release resources still exists. What should we do? "Duck typing" comes. You can manually write a Dispose() method, without inheriting any interfaces:

void S1Demo()
{
    using S1 s1 = new S1();
}

ref struct S1
{
    public void Dispose()
    {
        Console.WriteLine("Normal release");
    }
}

The same principle holds if using the IAsyncDisposable interface:

async Task S2Demo()
{
    await using S2 s2 = new S2();
}

struct S2 : IAsyncDisposable
{
    public async ValueTask DisposeAsync()
    {
        await Task.Delay(1);
        Console.WriteLine("Async release");
    }
}

8. T?, with Nullable<T> type

It's "black magic". Only Nullable<T> can accept T?. Nullable<T> is a value type, yet it can directly accept null values (normally, value types do not allow null values).

Example code is as follows:

int? t1 = null;
Nullable<int> t2 = null;
int t3 = null; // Error CS0037: Cannot convert null to 'int' because it is a non-nullable value type

Generated code is as follows (int? and Nullable<int> are exactly the same, skipping the code that failed to compile):

IL_0000: nop
IL_0001: ldloca.s 0
IL_0003: initobj valuetype [System.Runtime]System.Nullable`1<int32>
IL_0009: ldloca.s 1
IL_000b: initobj valuetype [System.Runtime]System.Nullable`1<int32>
IL_0011: ret

9. Generic operations of Index/Range for any type

There is "black magic", and there is "duck typing"—there is room for operation.

Index/Range was released with C# 8.0, which can conveniently operate on an index position and take out the corresponding value like Python. Things that required calling Substring and other complex operations are now very simple.

string url = "https://www.super-cool.com/product/7705a33a-4d2c-455d-a42c-c95e6ac8ee99/summary";
string productId = url[35..url.LastIndexOf("/")];
Console.WriteLine(productId);

Generated code is as follows:

string url = "https://www.super-cool.com/product/7705a33a-4d2c-455d-a42c-c95e6ac8ee99/amd-r7-3800x";
int num = 35;
int length = url.LastIndexOf("/") - num;
string productId = url.Substring(num, length);
Console.WriteLine(productId); // 7705a33a-4d2c-455d-a42c-c95e6ac8ee99

As you can see, the C# compiler ignored Index/Range and translated it directly into calling Substring.

But arrays are different:

var range = new[] { 1, 2, 3, 4, 5 }[1..3];
Console.WriteLine(string.Join(", ", range)); // 2, 3

Generated code is as follows:

int[] range = RuntimeHelpers.GetSubArray<int>(new int[5]
{
    1,
    2,
    3,
    4,
    5
}, new Range(1, 3));
Console.WriteLine(string.Join<int>(", ", range));

It can be seen that it really created a Range type and then called RuntimeHelpers.GetSubArray<int>, which is completely "black magic".

But it's also "duck typing". As long as the code implements the Length property and Slice(int, int) method, you can call Index/Range:

var range2 = new F()[2..];
Console.WriteLine(range2); // 2 -> -2

class F
{
    public int Length { get; set; }
    public IEnumerable<int> Slice(int start, int end)
    {
        yield return start;
        yield return end;
    }
}

Generated code is as follows:

F f = new F();
int length2 = f.Length;
length = 2;
num = length2 - length;
string range2 = f.Slice(length, num);
Console.WriteLine(range2);

Conclusion

As you can see, there are indeed a lot of "black magic" in C#, but there are also a lot of "duck typing", and there is a great "room for operation" for "tricks".

It is rumored that C# 9.0 will add "duck typing" tuples—"Type Classes". At that time, the "room for operation" will definitely be much larger than it is now, looking forward to it!