Zero-Width Characters

fabulous.yap — Wed, 11 Apr 2018 11:54:32 +0000

(Note: all credit goes to original author(s) linked within the text. the converted C# code is public domain, feel free to have fun with it)

Read an interesting article by Tom from Medium on using Zero-Width Characters as a technique to fingerprinting text, although it does not always work (depending on which editors/word processor you are using), it is interesting none-the-less. Tom has even created a demo website to demonstrate the techniques.

You can access his JS source code at ==> https://github.com/umpox/zero-width-detection.

I found the JS code interesting and wanted to try it with C# and offline. As a user of LINQPad, I'm fond of using it to create tiny offline routines for quick (and sometimes dirty) utils for my occasional needs. Following is the C# source.

using System;
using System.Text;
using System.Linq;

namespace zerowidth
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");
            string userName = "名字.Yap";
            string binaryText = string.Join(" ", Encoding.Unicode.GetBytes(userName).Select(c => Convert.ToString(c, 2).PadLeft(8, '0')).ToArray());
            Console.WriteLine(binaryText);

            string zeroWidthText2 = string.Join("\ufeff", binaryText.ToCharArray().Select(c => {  // Zero Width No-Break Space &#65279
                if (c == '1') return '\u200B';          // Zero Width Space &#8203
                else if (c == '0') return '\u200C';     // Zero Width Non-Joiner &#8204
                return '\u200D';                        // Zero Width Joiner &#8205
            }).ToArray());
            Console.WriteLine("|{0}|{1}", zeroWidthText2, zeroWidthText2.Length);

            string binaryText3 = string.Join("", zeroWidthText2.ToCharArray().Select(c => {
                if (c == '\u200B') return '1';
                else if (c == '\u200C') return '0';
                else if (c == '\u200D') return ' ';
                else return (char?)null;
            }).ToArray());
            Console.WriteLine(binaryText3);
            string originalUserName2 = Encoding.Unicode.GetString(binaryText3.Split(' ').Select(s => Convert.ToByte(s, 2)).ToArray());
            Console.WriteLine(originalUserName2);
        }
    }
}

Do follow and read the linked article for further details, Tom (and Zach Aysan, linked in Tom's article) has done an excellent job of elaborating and explaining the techniques.

It's been a long time since my last steganography in image processing and document fingerprinting, this has re-ignite the interest somewhat... :)

Chaining C# Assignment Operators

fabulous.yap — Mon, 26 Mar 2018 16:01:32 +0000

It all started with swap(), a simple programming question added to my interviewing question bank. In its simplest form, a temp variable is introduced as a place holder while we swap two Integer numbers:

static void swap(ref int a, ref int b)
{
  int tmp;

  tmp = a;
  a = b;
  b = tmp;
}

and of course, we have the following way of writing swap() without using a temporary variable:

static void swap(ref int a, ref int b)
{
  a = a ^ b;
  b = a ^ b;
  a = a ^ b;
}

which we can further simplify it using Assignment Operator to:

static void swap(ref int a, ref int b)
{
  a ^= b;
  b ^= a;
  a ^= b;
}

now, being a nerd coming from C/C++, we know we can shorten the above into:

/* C using pointer */
void swap(int *a, int *b)
{
  *a ^= *b ^= *a ^= *b;
}

in C language or C++...

// C++ using reference
void swap(int& a, int& b)
{
  a ^= b ^= a ^= b;
}

using Assignment Operator. These works as Assignment Operators are evaluated in right-associative manner... and here comes the equivalent C# codes...

static void swap(ref int a, ref int b)
{
  a ^= b ^= a ^= b;
}

but I was surprised that the above C# codes doesn't work as expected. Try the following program and you'll see it doesn't really swap the 2 integers...

using System;

namespace AssignmentOperatorChaining
{
    class Program
    {
        static void Main(string[] args)
        {
            int x = 0, y = 0;

            x = 10; y = 20;
            Console.WriteLine("x: {0},    y: {1}", x, y);
            swap(ref x, ref y);
            Console.WriteLine("x: {0},    y: {1}", x, y);
        }

        static void swap(ref int a, ref int b)
        {
            a ^= b ^= a ^= b;
        }
    }
}

the result is x: 0, y: 10 instead of x: 20, y: 10 after swap() was called. With some googling, I failed to find much information to this behavior.
Furthermore, a small change to the Assignment chaining code above works:

static void swap(ref int a, ref int b)
{
  b ^= a ^= b;
  a ^= b;
}

At this point, I'm really puzzled... but since .Net disassemblers are widely available, decided to take a peek under the hood to see what's going on. Created the following program to see the differences between a few different way of swapping 2 integers...

using System;

namespace AssignmentOperatorChaining
{
    class Program
    {
        static void Main(string[] args)
        {
            int x = 0, y = 0;

            x = 10; y = 20;
            Console.WriteLine("x: {0},    y: {1}", x, y);
            swap1(ref x, ref y);
            Console.WriteLine("x: {0},    y: {1}", x, y);

            x = 10; y = 20;
            Console.WriteLine("x: {0},    y: {1}", x, y);
            swap2(ref x, ref y);
            Console.WriteLine("x: {0},    y: {1}", x, y);

            x = 10; y = 20;
            Console.WriteLine("x: {0},    y: {1}", x, y);
            swap3(ref x, ref y);
            Console.WriteLine("x: {0},    y: {1}", x, y);

            x = 10; y = 20;
            Console.WriteLine("x: {0},    y: {1}", x, y);
            swap4(ref x, ref y);
            Console.WriteLine("x: {0},    y: {1}", x, y);

            x = 10; y = 20;
            Console.WriteLine("x: {0},    y: {1}", x, y);
            swap5(ref x, ref y);
            Console.WriteLine("x: {0},    y: {1}", x, y);
            Console.ReadLine();
        }

        static void swap1(ref int a, ref int b)
        {
            int tmp;
            tmp = a;
            a = b;
            b = tmp;
        }
        static void swap2(ref int a, ref int b)
        {
            a = a ^ b;
            b = a ^ b;
            a = a ^ b;
        }
        static void swap3(ref int a, ref int b)
        {
            a ^= b;
            b ^= a;
            a ^= b;
        }
        static void swap4(ref int a, ref int b)
        {
            b ^= a ^= b;
            a ^= b;
        }
        static void swap5(ref int a, ref int b)
        {
            a ^= b ^= a ^= b;
        }
    }
}

and the resulting il...

very interesting output. and what's more interesting is when feeding the exe to .Net disassembler like JustDecompile, the reverse engineer C# code for swap5() actually works:

private static void swap(ref int a, ref int b)
{
  int num = a ^ b;
  int num1 = num;
  a = num;
  int num2 = b ^ num1;
  num1 = num2;
  b = num2;
  a ^= num1;
}

(I've also tested with JetBrains' dotPeek, however, the disassembled output of swap5()triggers a panic mode in dotPeek and resulted in uncompile C# codes.)

if we take a closer look, there is really no difference between swap2() and swap3(), and it appears swap1() resulted in the shortest IL code and this can also be confirm when running in disassembling x86 mode:

regardless of whether we consider the a ^= b ^= a ^= b Assignment Operator Chaining is a C# compiler bug (tests has also been performed using .Net Core 2.1 in both Windows and Linux, same results), the moral of the story here is sometimes, the simplest thing could often be the best and/or fastest -- K.I.S.S. not to mention at the same time swap1() is more human readable too.

this Assignment Operator Chaining strange behavior not only affects ^= operator, others are affected too, here is a small program to test:

using System;

namespace AssignmentOperatorsChaining
{
    class Program
    {
        static void Main(string[] args)
        {
            int x=0, y=0;

            Console.WriteLine("-----------========== ^= operator ===========-----------");
            Console.WriteLine("chaining:");
            x = 10; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x ^= y ^= x ^= y;
            Console.WriteLine("x: {0},    y:{1}", x, y);

            Console.WriteLine("non-chaining:");
            x = 10; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x ^= y;
            y ^= x;
            x ^= y;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            Console.WriteLine("");

            Console.WriteLine("-----------========== *= operator ===========-----------");
            Console.WriteLine("chaining:");
            x = 10; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x *= y *= x *= y;
            Console.WriteLine("x: {0},    y:{1}", x, y);

            Console.WriteLine("non-chaining:");
            x = 10; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x *= y;
            y *= x;
            x *= y;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            Console.WriteLine("");

            Console.WriteLine("-----------========== += operator ===========-----------");
            Console.WriteLine("chaining:");
            x = 10; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x += y += x += y;
            Console.WriteLine("x: {0},    y:{1}", x, y);

            Console.WriteLine("non-chaining:");
            x = 10; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x += y;
            y += x;
            x += y;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            Console.WriteLine("");

            Console.WriteLine("-----------========== -= operator ===========-----------");
            Console.WriteLine("chaining:");
            x = 10; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x -= y -= x -= y;
            Console.WriteLine("x: {0},    y:{1}", x, y);

            Console.WriteLine("non-chaining:");
            x = 10; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x -= y;
            y -= x;
            x -= y;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            Console.WriteLine("");

            Console.WriteLine("-----------========== /= operator ===========-----------");
            Console.WriteLine("chaining:");
            x = 20; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x /= y /= x /= y;
            Console.WriteLine("x: {0},    y:{1}", x, y);

            Console.WriteLine("non-chaining:");
            x = 20; y = 20;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            x /= y;
            y /= x;
            x /= y;
            Console.WriteLine("x: {0},    y:{1}", x, y);
            Console.ReadKey();
        }
    }
}

Last but not least, if this has been documented/discussed somewhere before, or I've made mistakes in the codes, would greatly appreciate the collective wisdom of dev.to community to point me to the right direction. Thank you and good day to all.

DEV Community: fabulous.yap

Zero-Width Characters

(Note: all credit goes to original author(s) linked within the text. the converted C# code is public domain, feel free to have fun with it)

Chaining C# Assignment Operators