How text becomes binary: ASCII vs UTF-8, byte by byte

#beginners #computerscience #ascii #unicode

When you type a letter, your computer stores a number. Binary is just the way that number is written down. Here is the whole path from text to bits, with a small decoder you can run yourself.

Step 1: a letter is a number

ASCII (RFC 20) gives every basic letter, digit and punctuation mark a number from 0 to 127. H is 72 and i is 105. Write each number in 8 bits and you get:

H  72   01001000
i  105  01101001

So 01001000 01101001 reads as "Hi". A capital A is 65, or 01000001, and the word HELLO is 01001000 01000101 01001100 01001100 01001111.

Step 2: UTF-8 covers everything else

ASCII has no é and no emoji. UTF-8 (RFC 3629) keeps ASCII exactly as it is, one byte per character, and uses 2 to 4 bytes for the rest:

A   01000001
é   11000011 10101001
😀  11110000 10011111 10011000 10000000

That is why "Héllo" is 6 bytes, not 5. The leading bits of each byte tell a decoder how many bytes belong to the character.

Step 3: a decoder in a few lines

function binaryToText(input) {
  const bytes = input.trim().split(/\s+/).map((b) => parseInt(b, 2));
  return new TextDecoder("utf-8").decode(new Uint8Array(bytes));
}

function textToBinary(text) {
  return [...new TextEncoder().encode(text)]
    .map((b) => b.toString(2).padStart(8, "0"))
    .join(" ");
}

binaryToText("01001000 01101001"); // "Hi"
textToBinary("é");                 // "11000011 10101001"
binaryToText(textToBinary("Héllo 😀")); // "Héllo 😀"

Common mistakes

Dropped leading zeros. 1001000 is 7 bits. Pad every byte to 8 bits or you will shift everything after it.
A stray space. The byte 00100000 is a space character, so an extra one shows up as a blank in your output.
Assuming one byte per character. That holds for ASCII only. Accents and emoji take 2 to 4 bytes.

Checking your own examples

To check a message without writing code, paste it into a binary code translator. It detects 8-bit, 7-bit and UTF-8 binary, and works in both directions.

Disclosure: I built it. It is part of The Code Translator, a free set of decoders that run in your browser without sign-up.

What would you add to this? I would like to hear how you debug encoding problems in your own projects.

DEV Community