jack

Posted on Sep 22

A Binary String Has No Meaning Until You Choose an Encoding

#javascript #unicode #webdev #tutorial

Suppose someone sends you this:

What does it mean?

It is tempting to paste it into a converter and accept the first readable answer. But the honest answer is: the bits alone do not tell us.

Depending on the format, 00101 can be the number 5, the letter E, the letter S, or an incomplete text byte. A decoder does not discover meaning hidden inside the digits. It applies rules supplied by a format.

That distinction sounds small, but it explains many confusing binary conversions.

One bit string, several valid interpretations

Here are four ways to read 00101:

Interpretation	Result	Assumption
Unsigned integer	5	Read all five bits as a base-2 number
Five-bit A1Z26	E	Use the teaching convention `00001 = A` through `11010 = Z`
International ITA2	S	Use ITA2 numeric display in Letters state
ASCII or UTF-8 text	Incomplete	Text decoding needs a complete seven-bit ASCII group or eight-bit byte

None of these results changes the original digits. Only the interpretation changes.

This is the same idea programmers use every day with types. A byte in memory is not inherently a character, an unsigned number, part of a floating-point value, or a color channel. The program decides what the byte represents.

Bits provide syntax; formats provide meaning

To decode an unknown binary string, we need more than zeros and ones. At minimum, we need to know:

Group width: Are values split into 5, 7, 8, 16, or another number of bits?
Encoding: Is the source ASCII, UTF-8, ITA2, or a custom scheme?
Numeric rules: Is the value unsigned, signed, fixed-point, or something else?
Framing: Where does one value end and the next begin?
Context: Did the bits come from text, a sensor, a network packet, an image, or a puzzle?

Consider this sequence:

0100100001101001

If the source says it is UTF-8 text, we can split it into bytes:

01001000 01101001

Those bytes have decimal values 72 and 105, which decode to H and i.

But if the source says it is one unsigned integer, the same 16 bits represent 18,537. The digits did not change. The framing and data type did.

ASCII and UTF-8 agree only within the ASCII range

The original ASCII standard defines 128 values, so it requires seven bits. ASCII is often stored in an eight-bit byte with a leading zero.

For example, uppercase A is decimal 65:

7-bit ASCII:         1000001
8-bit storage:      01000001
UTF-8:              01000001

UTF-8 deliberately uses the same byte values for the ASCII range. That is why ordinary English letters often decode correctly whether a tool labels the mode ASCII or UTF-8.

Outside that range, the difference becomes important. The character é is encoded as two UTF-8 bytes:

11000011 10101001

Those bytes are decimal 195 and 169, but they should not be decoded as two unrelated characters. Together they encode Unicode code point U+00E9.

Likewise, having eight bits does not guarantee that a value is valid text:

Bits	Unsigned integer	Strict ASCII	Strict UTF-8
`01000001`	65	A	A
`00001010`	10	Line feed	Line feed
`11111111`	255	Outside ASCII	Invalid UTF-8 byte

A decoder that always produces a visible character is often hiding an assumption or replacing invalid data.

Decode UTF-8 strictly in JavaScript

JavaScript already provides the right primitives for browser-side UTF-8 conversion: TextEncoder and TextDecoder.

Here is a small strict decoder:

function decodeUtf8Binary(input) {
  const trimmed = input.trim();

  if (!trimmed) return "";

  if (/[^01\s,]/.test(trimmed)) {
    throw new Error(
      "Use only 0, 1, spaces, commas, or line breaks."
    );
  }

  const bits = trimmed.replace(/[\s,]/g, "");

  if (bits.length % 8 !== 0) {
    const missing = 8 - (bits.length % 8);
    throw new Error(
      `The final byte is incomplete. Add ${missing} more bit(s).`
    );
  }

  const bytes = bits
    .match(/.{8}/g)
    .map(group => Number.parseInt(group, 2));

  return new TextDecoder("utf-8", {
    fatal: true,
    ignoreBOM: true
  }).decode(new Uint8Array(bytes));
}

console.log(
  decodeUtf8Binary("01001000 01101001")
); // Hi

console.log(
  decodeUtf8Binary("11000011 10101001")
); // é

The important option is fatal: true.

Without it, malformed input may be replaced with the Unicode replacement character �. That is convenient for displaying damaged text, but misleading in a converter: it makes invalid bytes look like a successful, if slightly strange, result.

For the reverse direction, TextEncoder returns the exact UTF-8 bytes:

function encodeUtf8Binary(text) {
  return Array.from(new TextEncoder().encode(text))
    .map(byte => byte.toString(2).padStart(8, "0"))
    .join(" ");
}

console.log(encodeUtf8Binary("Hi"));
// 01001000 01101001

console.log(encodeUtf8Binary("é"));
// 11000011 10101001

This also demonstrates why “one character equals one byte” is not a safe rule for Unicode text.

Five bits do not identify one universal alphabet

Five-bit puzzles create another common source of confusion.

A classroom may use an A1Z26-style convention:

00001 = A
00010 = B
00011 = C
...
11010 = Z

Under that convention, 00101 means E.

ITA2, the historical teleprinter alphabet, uses a different table. In its Letters state, numeric-display 00101 means S. ITA2 also has shift codes that switch between letters and figures, so the meaning of a later group can depend on earlier groups. Serial transmission order can reverse how the five bits are displayed as well.

Therefore, “five bits” is a group width, not a complete encoding specification.

Padding is an assumption, not a repair

Suppose the original input is:

As an unsigned integer, it is unambiguously 5.

Adding leading zeroes preserves that numeric value:

101      = 5
00101    = 5
00000101 = 5

But the added width creates new possible interpretations:

00101 can be E under the five-bit A1Z26 convention.
00000101 is the control value ENQ in ASCII, not E.

Padding did not reveal a hidden letter. It introduced a field-width assumption.

This matters even more in a stream. Adding or removing one bit can move every following byte boundary. A decoder should preserve the original input and explain any padding instead of silently changing it until readable text appears.

Valid output can still be wrong

Successful decoding proves that the bytes are valid under a format. It does not prove that the bits were copied correctly.

For example:

01001000 01101001 = Hi
01001000 01101000 = Hh

Only the final bit changed. Both results are valid ASCII and valid UTF-8.

This is especially important when binary digits come from OCR. A confidence score can highlight areas worth checking, but it cannot certify the message. A one-bit recognition error may still produce perfectly valid text.

A practical decoding checklist

When a binary result looks surprising, check these in order:

Preserve the original sequence before editing anything.
Identify the source: text, integer, image, packet, or known historical code.
Confirm the expected group width.
Keep leading zeroes when they define text boundaries.
Select the encoding explicitly.
Reject incomplete groups instead of guessing missing bits.
Use strict UTF-8 decoding when accuracy matters.
Inspect control characters when the result appears blank.
For OCR input, compare every recognized digit with the image.
Treat readable output as evidence, not proof.

Making the assumptions visible

I wanted a converter that did not hide these distinctions, so I built Binary Code Translator. It can compare UTF-8 and strict ASCII text, exact unsigned integers, A1Z26, and international ITA2, while keeping the selected interpretation visible.

The larger lesson applies well beyond binary puzzles:

Data does not explain itself. Meaning comes from a format, and a reliable tool makes that format explicit.

DEV Community