DEV Community: Mahmoud Ibrahim — Confileo

I printed Arabic to PDF with 13 fonts. Only one produced text you can search

Mahmoud Ibrahim — Confileo — Wed, 23 Sep 2026 22:16:00 +0000

If you generate PDFs from HTML with Puppeteer, Playwright or headless Chrome, and any of your users write in Arabic, there is a good chance your PDFs contain no searchable Arabic text. They look perfect. You cannot tell by opening them. You find out when a user says "I can't copy the name out of the invoice", or when an applicant tracking system rejects a CV that reads fine to a human.

I ran into this while building a text-to-PDF feature, and then measured it properly: 13 fonts, the same Arabic paragraph, printed by headless Chrome, read back with pdf.js. One font passed.

The test

The document is four short lines: a title (خطاب تعريف), a sentence that contains ordinary words such as الموظف, a line that mixes Arabic with Latin text and digits, and three words chosen because they contain the lam-alef ligature: الأمر، الآن، لإدارة.

Each font is embedded with @font-face, the page is printed with page.pdf(), and the PDF goes straight into pdfjs-dist to get the text layer. Three checks:

Presentation forms. Are the stored characters real Arabic letters (U+0600–U+06FF), or the Unicode presentation forms (U+FB50–FDFF, U+FE70–FEFF) that describe a letter's shape instead of the letter?
Lam-alef order. Do الأمر / الآن / لإدارة come back as typed?
Whole words. How many of the 18 Arabic words can a plain text search find?

The results

Chrome 135, pdf.js 4.8, September 2026:

Font	Stored as	لا ligature	Words found
Amiri Regular	real letters	in order	15 / 18
Amiri Bold	real letters	in order	16 / 18
IBM Plex Sans Arabic	presentation forms	—	0 / 18
Noto Sans Arabic	presentation forms	—	0 / 18
Arial	presentation forms	—	0 / 18
Tahoma	presentation forms	—	0 / 18
Times New Roman	presentation forms	—	0 / 18
Segoe UI	presentation forms	—	1 / 18
Traditional Arabic	presentation forms	—	0 / 18
Simplified Arabic	presentation forms	—	0 / 18
Arabic Typesetting	presentation forms	—	0 / 18
Sakkal Majalla	presentation forms	—	1 / 18
Andalus	presentation forms	—	0 / 18

An earlier run with web fonts told the same story. Readex Pro, Scheherazade New and Lateef stored real letters but reversed the lam-alef ligature; Noto Naskh Arabic, Noto Kufi Arabic, Cairo, Almarai and Tajawal stored presentation forms.

The two or three words Amiri "misses" are not a font problem: pdf.js sometimes puts a space inside a word where one text run ends and the next begins. All the letters are there and in order.

Why this happens

Arabic is cursive. Most letters have up to four shapes depending on their neighbours, and some pairs fuse into a single glyph. Chrome shapes the text with HarfBuzz and writes the resulting glyphs into the PDF. For the text to be extractable, the PDF also needs a ToUnicode map: glyph number → the characters it represents.

For most Arabic fonts, Chrome cannot trace a shaped glyph back to the character that produced it, so the map points at the presentation-form codepoint for that shape. The file renders correctly, because rendering uses the glyphs. Extraction uses the map, so what you copy is a different string from what you typed. Search for الموظف and you are searching for letters that are not in the file.

The ligature is the second trap. لا is one glyph standing for two letters, lam followed by alef. Several fonts get every other letter right and then map that glyph back as alef + lam, reversed. That breaks every word where ال is followed by أ, إ or آ, which in Arabic is a lot of words.

Check your own font

I put the test in a small repo so it can run in CI next to your templates:

git clone https://github.com/mahmoudQq2023/arabic-pdf-fonts
cd arabic-pdf-fonts && npm install
node check-font.mjs ./fonts/YourFont-Regular.ttf ./fonts/YourFont-Bold.ttf

YourFont-Regular.ttf   BROKEN: presentation forms (text not searchable)  (0/18 words searchable)
Amiri-Regular.ttf      OK: real letters, lam-alef in order  (15/18 words searchable)

It exits with code 1 if any font fails, and --json gives you something to assert on. The results page is at mahmoudqq2023.github.io/arabic-pdf-fonts.

What I changed in production

Amiri is embedded for every Arabic document, including a real Bold file. Synthetic bold draws each glyph as its own run, which can split a bold heading into one run per letter.
No font picker. Offering Arabic fonts that look nicer but produce unsearchable files felt like a trap for users who will never know why their CV is not being parsed.
No promise about harakat. Diacritics (U+064B–0652) rendered correctly in every font and came back as U+0000 in all of them. Copy gives you the base letters.
A round-trip test in the repo. The page never shows this bug, so only a test that reads the PDF back will catch a regression when someone swaps a font later.

The feature this came from is Confileo's text to PDF converter. If you only need to turn Arabic text into a searchable PDF, you can use it directly; if you generate PDFs yourself, run the checker on your fonts before your users find the problem for you.

Spelling numbers in Arabic is harder than it looks: the four rules behind tafqeet

Mahmoud Ibrahim — Confileo — Tue, 22 Sep 2026 22:03:39 +0000

Most "number to words" libraries translate digits one at a time. That works in English and fails in Arabic on the very first cheque. I maintain an Arabic number-to-words converter (تفقيط) used for invoices and cheques, and these are the rules that broke every naive version I wrote.

1. Units come before tens

English says twenty-five. Arabic says five and twenty: خمسة وعشرون. Any loop that walks the digits left to right and concatenates words produces «عشرون وخمسة», which a bank clerk reads as wrong at a glance.

2. Three to ten take the opposite gender of the noun

This is the rule everyone forgets. One and two agree with the counted noun; three to ten take the other gender:

count	masculine noun (جنيه)	feminine noun (ليرة)
1	جنيه واحد	ليرة واحدة
3	ثلاثة جنيهات	ثلاث ليرات
10	عشرة جنيهات	عشر ليرات

So the words for 3–10 depend on the currency, not only on the digit. A converter that hard-codes «ثلاثة» is correct for pounds and wrong for lira.

3. The counted noun changes form with the last two digits

This is the part that makes a lookup table impossible:

last two digits of the amount	form of the noun	example
3–10	plural	ثلاثة آلاف، سبعة جنيهات
11–99	singular, accusative (tanween)	أحد عشر ألفاً، خمسون جنيهاً
00 (100, 1,000…)	singular	مائة جنيه، ألف جنيه

The same rule applies to the scale words themselves, which is why 3,000 is «ثلاثة آلاف» but 11,000 is «أحد عشر ألفاً» and 100,000 is «مائة ألف».

4. Two hundred loses its ن in front of a noun

«مائتان» on its own, but مائتا ألف and مائتا جنيه when a noun follows (the dual in a construct loses its final nun). A converter that emits «مائتان ألف» is one character off, and that character is exactly what a proof-reader looks for.

A version that gets these right

const ONES = ['', 'واحد', 'اثنان', 'ثلاثة', 'أربعة', 'خمسة', 'ستة', 'سبعة', 'ثمانية', 'تسعة'];
const TEENS = ['عشرة', 'أحد عشر', 'اثنا عشر', 'ثلاثة عشر', 'أربعة عشر', 'خمسة عشر',
               'ستة عشر', 'سبعة عشر', 'ثمانية عشر', 'تسعة عشر'];
const TENS = ['', '', 'عشرون', 'ثلاثون', 'أربعون', 'خمسون', 'ستون', 'سبعون', 'ثمانون', 'تسعون'];
const HUNDREDS = ['', 'مائة', 'مائتان', 'ثلاثمائة', 'أربعمائة', 'خمسمائة',
                  'ستمائة', 'سبعمائة', 'ثمانمائة', 'تسعمائة'];

// 1..999, masculine, nominative (the cheque convention)
function below1000(n) {
  const parts = [];
  const h = Math.floor(n / 100), r = n % 100;
  if (h) parts.push(HUNDREDS[h]);
  if (r >= 20) {
    const u = r % 10, t = Math.floor(r / 10);
    parts.push(u ? `${ONES[u]} و${TENS[t]}` : TENS[t]);   // units before tens
  } else if (r >= 10) parts.push(TEENS[r - 10]);
  else if (r) parts.push(ONES[r]);
  return parts.join(' و');
}

// Pick the noun form from the last two digits (rule 3), and fix مائتان (rule 4).
function counted(k, forms) {
  if (k === 1) return forms.one;
  if (k === 2) return forms.two;
  const r = k % 100;
  const noun = r >= 3 && r <= 10 ? forms.few : r >= 11 ? forms.many : forms.one;
  const words = r === 0 && Math.floor(k / 100) % 10 === 2 && k < 1000 ? 'مائتا' : below1000(k);
  return `${words} ${noun}`;
}

const THOUSAND = { one: 'ألف', two: 'ألفان', few: 'آلاف', many: 'ألفاً' };
const MILLION = { one: 'مليون', two: 'مليونان', few: 'ملايين', many: 'مليوناً' };

function tafqeet(n) {
  if (n === 0) return 'صفر';
  const m = Math.floor(n / 1e6), t = Math.floor(n / 1e3) % 1000, rest = n % 1000;
  const parts = [];
  if (m) parts.push(counted(m, MILLION));
  if (t) parts.push(counted(t, THOUSAND));
  if (rest) parts.push(below1000(rest));
  return parts.join(' و');
}

tafqeet(25);        // خمسة وعشرون
tafqeet(3000);      // ثلاثة آلاف
tafqeet(11000);     // أحد عشر ألفاً
tafqeet(200000);    // مائتا ألف
tafqeet(1250750);   // مليون ومائتان وخمسون ألفاً وسبعمائة وخمسون

That covers the integer part in the masculine. A real cheque line still needs the currency noun chosen by rule 3, the feminine table for feminine currencies, the fraction («وخمسون قرشاً»), and the framing banks expect: فقط ... لا غير.

Try it on your own numbers

The code above is open source as arabic-tafqeet (MIT, one file, tests included). The full version — pound, riyal, dirham, dinar and dollar, the fraction, the فقط/لا غير framing, and words back to digits — runs as a free converter: تفقيط الأرقام — Arabic number to words for cheques and invoices. There is an English number to words tool on the same site if you need both languages on one invoice.

If you find a number it spells wrong, tell me in the comments — that is how every rule above was found.

Por qué el texto invisible funciona en el nombre de TikTok y falla en el usuario de Instagram

Mahmoud Ibrahim — Confileo — Wed, 09 Sep 2026 20:11:19 +0000

Cada tanto vuelve la misma pregunta con otro disfraz: cómo poner el nombre invisible en TikTok, cómo mandar un mensaje vacío en WhatsApp. En 2026 la de moda es la de TikTok, y casi ningún video explica el mecanismo. Va aquí, desde el código.

Un espacio no es un carácter: para la app es un separador

Cuando escribes un nombre o un mensaje, la app lo recorta antes de guardarlo: trim() quita los espacios de los extremos, las secuencias se colapsan y un campo con solo espacios queda vacío. Después corre la validación: nombre vacío, rechazado.

Por eso el texto invisible no puede estar hecho de espacios, sino de caracteres que se dibujan como nada, pero que para el código de recorte no son espacio en blanco. Unicode tiene varios, ninguno diseñado para esto, y cada app los trata distinto.

Los cinco que importan

Carácter	Code point	Para qué existe	Quién lo conserva
Hangul Filler	`U+3164`	Relleno de sílabas coreanas: una "letra" que no dibuja nada	Nombre y bio de TikTok e Instagram; nick de Free Fire y PUBG Mobile
Braille Pattern Blank	`U+2800`	La celda braille sin puntos: un glifo imprimible y vacío	Mensajes de WhatsApp, Telegram y Discord; comentarios de Instagram
Zero-Width Space	`U+200B`	Punto de corte de línea sin ancho	X, Notas de iPhone, HTML; la mayoría de los chats lo elimina
Invisible Separator	`U+2063`	Límite invisible entre elementos	Facebook y Messenger
Hair Space	`U+200A`	El espacio tipográfico más delgado	Word, Google Docs, correo

Hangul Filler es una letra (categoría Lo): un validador que pregunta "¿tiene al menos una letra?" responde que sí. Braille Blank es un símbolo (So), imprimible, así que "el mensaje no puede estar vacío" pasa. Zero-Width Space es un carácter de formato (Cf), y los caracteres de formato son lo primero que borra un sanitizador; por eso falla en WhatsApp y funciona en un editor de texto.

El truco de TikTok, explicado en un párrafo

El "nombre invisible en TikTok 2026" funciona así: el nombre para mostrar de TikTok es un campo de texto libre que valida "no vacío", no "solo letras y números". Pegas un U+3164, el campo recibe una letra, trim() no la toca porque no es espacio en blanco, y el servidor guarda un nombre que el renderizador dibuja como nada. Lo único nuevo frente a los nicks de Free Fire es el video. El mismo carácter funciona en el nombre y en las líneas de la bio de Instagram, pero no en el nombre de usuario (@handle): ese campo acepta únicamente letras, dígitos, puntos y guiones bajos, y ahí no hay blanco que sobreviva.

Por qué un nombre invisible se rompe tras una actualización

Un juego que antes aceptaba U+3164 y ahora lo rechaza no "arregló los nombres invisibles": alguien agregó normalización Unicode o un filtro por categoría al validador, casi siempre para frenar suplantaciones con nombres idénticos a la vista. En cuanto corre NFKC, U+3164 se convierte en U+1160 (Hangul Jungseong Filler), y un filtro por categoría lo elimina del todo; U+FFA0 (la forma de medio ancho) se convierte en lo mismo. Roblox es el ejemplo típico: elimina la mayoría de los rellenos tras cada actualización. Y como en Free Fire cambiar de nombre cuesta 390 diamantes o una tarjeta, prueba en una cuenta secundaria antes de gastar.

Detectarlo es trivial

Invisible no significa oculto: cualquier cosa que cuente caracteres lo delata:

const INVISIBLES = new Set([0x200B, 0x200C, 0x200D, 0x2800, 0x3164, 0xFFA0, 0x2063, 0x200A]);

const reporte = (s) => {
  let invisibles = 0;
  for (const ch of s) if (INVISIBLES.has(ch.codePointAt(0))) invisibles++;
  return { total: [...s].length, invisibles, utf16: s.length };
};

reporte('anaㅤ⠀');   // { total: 5, invisibles: 2, utf16: 5 }
reporte('👨‍👩‍👧');    // { total: 5, invisibles: 2, utf16: 8 }

Dos detalles que muerden. .length cuenta unidades UTF-16, no caracteres: un emoji fuera del BMP vale 2 y el de la familia vale 8 (tres personas unidas por dos U+200D, que también caen en la lista: segmenta en grafemas con Intl.Segmenter antes de filtrar). Itera siempre con for…of. Y normaliza después de contar: s.normalize('NFKC') convierte U+3164 y U+FFA0 en U+1160 (que no está en la lista) y el espacio fino en un espacio normal: el contador deja de verlos.

Dónde conseguir el carácter correcto

Un teclado no tiene tecla para U+3164; se copia de una página que lo tenga listo. La versión en español de texto invisible para copiar y pegar ofrece los cinco caracteres de la tabla con la app que conserva cada uno. La referencia en inglés, con entidades HTML, escapes CSS y una casilla que comprueba si lo que pegaste contiene un carácter invisible o solo espacios, está en la página de invisible text. Todo corre en el navegador: es un navigator.clipboard.writeText de una cadena que la página ya contiene.

Resumen práctico

Mensaje en blanco en WhatsApp, Telegram o Discord: U+2800. WhatsApp y Telegram eliminan U+200B.
Nombre en TikTok, Instagram, Free Fire o PUBG Mobile, o línea vacía de la bio en TikTok e Instagram: U+3164 (respaldo: U+FFA0).
Publicación en blanco en X o línea oculta en Notas de iPhone: U+200B; iMessage lo rechaza si va solo, ahí usa U+2800.
Facebook y Messenger: U+2063. Documentos y correo: U+200A.
Nada funciona en un campo que acepta solo [a-z0-9._]: el nombre de usuario de Instagram y el de Discord.

Si mantienes un campo de nombre y quieres frenar esto: normaliza con NFKC y rechaza lo que quede vacío tras quitar \p{Cf}, \p{Zs} y los dos rellenos Hangul. Y dilo en el mensaje de error, porque "nombre inválido" manda a la gente directo a las páginas de arriba.

"Unix timestamp to Excel date: why serial 60 is a day that never existed"

Mahmoud Ibrahim — Confileo — Mon, 07 Sep 2026 01:00:41 +0000

You paste 1756771200 into a spreadsheet and want to see a date. Or you export a sheet and your API receives a column of numbers like 46236.5 instead of dates. Both directions look trivial and both go wrong in the same three places: the epoch, the unit, and one fake day in February 1900.

What the two numbers actually count

A Unix timestamp is the number of seconds since 1 January 1970, 00:00 UTC. Leap seconds are ignored, so every day is exactly 86,400 seconds long. A timestamp from today has 10 digits. If you see 13 digits it is milliseconds (what Date.now() returns in JavaScript). 16 digits is microseconds, 19 digits is nanoseconds (Go, Rust and several databases use those).

An Excel serial date is the number of days since Excel's epoch, with the fraction of the day as the decimal part. 46236.5 is noon on the day with serial 46236. Excel never stores a time zone; the number is whatever you typed.

The formulas

Unix seconds to Excel serial, then format the cell as a date:

=A1/86400 + 25569

25569 is the serial of 1 January 1970. For milliseconds divide by 86400000 instead.

Excel serial to Unix seconds:

=(A1 - 25569) * 86400

Both formulas produce UTC. Excel does not know where you are, so a timestamp from a server will show the UTC hour. Add + 2/24 to shift to UTC+2, - 5/24 for UTC-5, and so on. If a whole column is off by exactly the same number of hours, that is the cause, not the formula.

In a German, Spanish or French copy of Excel the arithmetic is identical, but the function names and separators are not: DATE(1970,1,1) becomes DATUM(1970;1;1), FECHA(1970;1;1) or DATE(1970;1;1) with semicolons. The formula above avoids the function entirely, which is why it travels between language versions without editing.

Serial 60: the day that never happened

Excel's default 1900 date system says serial 1 is 1 January 1900 and serial 60 is 29 February 1900. That date does not exist. 1900 is divisible by 100 but not by 400, so it was not a leap year. The bug comes from Lotus 1-2-3, and Excel reproduced it in 1987 so that spreadsheets imported from Lotus would keep their dates. It has been kept ever since, on purpose, because fixing it would move every date in every old file by one day.

The consequences are small but real:

Every serial below 61 is one day off against the real calendar.
DATE(1900,2,29) is accepted instead of raising an error.
WEEKDAY returns the wrong day for dates in January and February 1900.

This is also why libraries such as pandas use 30 December 1899 as "day zero" when they read Excel files: counting from there absorbs the phantom day, and every date from 1 March 1900 onward comes out right.

Google Sheets and LibreOffice count from 30 December 1899 without the fake leap day. Their serials agree with Excel from 1 March 1900 onward and differ by one before that. Older Mac versions of Excel used a 1904 date system (day zero is 1 January 1904). A workbook created there and opened elsewhere shows every date 1,462 days off, which is four years and one day. The fix is in File → Options → Advanced → "Use 1904 date system", not in the data.

The unit mistakes you will actually see

A 13-digit value in a seconds converter produces a date thousands of years in the future. Count the digits before you convert.
A 19-digit nanosecond stamp does not fit in a JavaScript number. Doubles hold integers exactly only up to 2^53, about 9.0 × 10^15, so the last digits get rounded. Treat it as a string and split off the last nine digits as the fraction of a second.
19 January 2038, 03:14:07 UTC is where a signed 32-bit seconds counter overflows. Anything still storing timestamps in an int32 will wrap to December 1901 at that moment.
Excel's own limit is serial 2,958,465, which is 31 December 9999. Anything above shows as #####.

Doing it without a formula

If you have a column of timestamps rather than one value, the Unix timestamp converter takes the whole column at once. It detects seconds, milliseconds, microseconds and nanoseconds from the digit count, prints UTC and your local time on separate lines, gives the Excel serial and the ISO 8601 form for each row, flags serials below 61 because of the phantom day above, and writes the formula in the language of the Excel you use. Nothing is uploaded; the arithmetic runs in the browser.

Going the other way, if what you have is a date and you need the number, the same page converts an Excel serial or an ISO date back to a timestamp in any of the four units.

A text diff that reads .docx and .pdf in the browser, and the trap that makes every PDF look 100% changed

Mahmoud Ibrahim — Confileo — Sat, 05 Sep 2026 11:38:09 +0000

I maintain a small online text diff that
people mostly use for contracts, price lists and CVs, not code. Its search traffic
told me something I had not designed for: the queries were "compare two Word
documents", "compare 2 text files online", "compare two PDF documents".
Nobody arrives holding two strings. They arrive holding two files.

This post is the handful of decisions that turned a textbook LCS exercise into
something that survives real documents. None of it is clever; all of it was
learned by watching the tool be wrong.

Two levels, not one

A diff over whole lines is what git diff shows you and it is the right first
level: it keeps paragraphs aligned, and a moved clause shows up as one removal
and one addition instead of a cloud of red and green words.

But a contract edit is usually a single number inside a long line. A line-only
diff paints the whole line red and the whole replacement green, and the reader
still has to find the number by eye — which is the exact task the tool exists
to remove.

So the second level runs only on lines the first level paired as "modified":

// Level 1: LCS over lines -> ops of keep / remove / add
const ops = computeLCS(leftLines, rightLines);

// Adjacent removes and adds are paired positionally as "modified" lines,
// and each pair gets a Level 2 diff over its words.
const pairs = Math.min(removes.length, adds.length);
for (let p = 0; p < pairs; p++) {
  const [lw, rw] = wordDiff(leftLines[removes[p]], rightLines[adds[p]]);
  result.push({ type: 'modified', leftWords: lw, rightWords: rw, ... });
}

The pairing rule is deliberately dumb: the k-th removed line in a block is
matched with the k-th added line. A smarter pairing (best similarity across
the block) sounds better and is worse in practice on prose, because a block of
three rewritten sentences is almost always three edits in order, and similarity
scoring will happily pair sentence 1 with sentence 3 because they share the
word "the".

Word tokenising keeps whitespace as its own tokens:

const tokens = text.match(/(\S+|\s+)/g) ?? [];

That looks pointless until you render: if you drop the spaces, you cannot
reconstruct the line, and a diff you cannot reconstruct is a diff you cannot
copy out.

LCS, not Myers, and the number that actually matters

Every diff article eventually says "use Myers' O(ND) algorithm". For a tool that
runs on someone's phone against two 40-page contracts, I did not, and here is
the honest reasoning.

The plain dynamic-programming LCS is O(m·n) in time and memory. Two 2,000-line
documents are 4 million cells — fine. Two 5,000-line exports are 25 million
cells of a number[][], which on a mid-range Android phone is a tab crash, not
a slow result. Myers fixes the typical case but its worst case is the same
shape, and the failure mode is identical: the browser dies with no message.

So the thing that made the tool reliable was not the algorithm. It was a guard:

if (m * n > 5_000_000) {
  return simpleDiff(left, right);   // positional, O(max(m, n)), always finishes
}

Above 5M cells it degrades to a line-by-line positional compare. That produces
a worse diff for large inputs with insertions near the top (everything after
the insertion is offset by one and reads as changed). But it produces one, in
milliseconds, and the visitor can narrow the selection and try again. A diff
tool that sometimes shows nothing is worth less than one that is sometimes
coarse. The same cap exists inside the word level at 500K cells, where the
fallback is "whole line removed, whole line added".

If your inputs are code and your users are on laptops, Myers is the right call.
If your inputs are documents and your users are on phones, put the guard in
first and pick the algorithm second.

Reading the files without a server

.docx and .xlsx are ZIP containers. .pdf is something every modern browser
already knows how to render. That means extraction can happen entirely
client-side, and for this particular tool that is not a nice-to-have: the
documents people most often need to compare are contracts, payslips, offers and
medical reports. A comparison that requires uploading both versions is not one
they can honestly run on those files.

Three things that bit me, in order of how much support mail they generated:

Scanned PDFs have no text. A scan is a picture of a page. The extraction
"succeeds" and returns an empty string, and a naive tool then reports the
document as identical to an empty box. Detect the empty result and say the word
OCR to the user, or they will assume the tool is broken.

Not everything is UTF-8. A .txt exported from an older Windows app arrives
in Windows-1252, and decoding it as UTF-8 turns every é and – into a
replacement character — which then shows up as a difference between two files
that are byte-for-byte the same. Fall back to Windows-1252 when UTF-8 decoding
produces replacement characters, and tell the user you did, so they check the
accented characters themselves.

Size. I cap at 8 MB per side. Not because the ZIP or PDF parsers cannot
handle more, but because the LCS guard above is line-count based, and an
enormous file is the one input that will blow past every guard at once.

The trap: PDF text is wrapped

This is the one I would put at the top of the page if I could only keep one
sentence.

Text extracted from a PDF carries a line break wherever the layout broke the
line — mid-sentence, after every 80-odd characters. Text from the .docx the PDF
was exported from carries a line break only at paragraph ends. Compare the two,
and a line-based diff reports every paragraph as changed, because no line
on the left is equal to any line on the right, even though not a word differs.

You can partially rescue it by re-joining lines that do not end in sentence
punctuation before you compare — a preprocessing step, and one I have not
automated, because it guesses wrong on lists, addresses and table cells. The
"ignore whitespace" option does something narrower and safer: it collapses
runs of spaces and tabs within a line, which fixes the double-space and
tab-versus-spaces noise a PDF exporter adds, but cannot undo a line break.
There is no fully general fix, because a PDF also breaks lines at the ends of
table cells and around headers and footers, and reflowing those back into
prose is a layout-analysis problem, not a string problem. The honest answer
on the page is: compare paragraph by paragraph rather than pasting the whole
document at once, and the output becomes far more useful.

What I would keep if I rewrote it

Two levels: lines first, words only inside modified pairs.
Positional pairing of removes with adds. Boring and right for prose.
A cell-count guard before any algorithm choice.
Client-side extraction, with explicit messages for the empty-scan and wrong-encoding cases.
One sentence about PDF line wrapping, above the fold.

The running tool is at
confileo.com/tools/text-diff if you
want to see how the modified-line rendering reads with real documents. It does
not store the text, which is the whole point when the documents are contracts.

"Invisible text is not empty: the five Unicode blanks apps actually keep"

Mahmoud Ibrahim — Confileo — Wed, 02 Sep 2026 07:09:03 +0000

Every few weeks someone asks the same question in a different costume: how do I send an empty message on WhatsApp, how do I make my Free Fire name invisible, why does my Instagram bio lose its blank lines. The answer is the same each time, and it is not a trick inside any of those apps. It is Unicode.

A space is not a character. To an app, it is a separator.

When you type a name, a message or a bio, the app trims it before saving. Leading and trailing whitespace goes, runs of spaces collapse, and a field that contains only spaces becomes an empty string. Then the validation runs: empty name, empty message, rejected.

So "invisible text" cannot be made of spaces. It has to be made of characters that render as nothing but are not whitespace to the trimming code. Unicode has several, and they were never designed for this. Each exists for a real typographic or scripting reason, which is exactly why every app treats them differently.

The five that matter, and what each was for

Character	Code point	Why it exists	Who keeps it
Hangul Filler	`U+3164`	A placeholder in Korean syllable composition — a "letter" that draws nothing	Game nickname fields (Free Fire, PUBG Mobile), Instagram name and bio, TikTok nickname
Braille Pattern Blank	`U+2800`	The braille cell with no dots raised — a printable glyph that is empty	WhatsApp, Telegram and Discord messages, Instagram comments
Zero-Width Space	`U+200B`	A line-break opportunity with no width, for scripts without spaces	X posts, Apple Notes, HTML; stripped by most chat apps
Invisible Separator	`U+2063`	Marks an invisible boundary between items (think matrix indices)	Facebook and Messenger
Hair Space	`U+200A`	The thinnest typographic space	Word, Google Docs, email

The pattern is not random. Hangul Filler is a letter (category Lo), so a name validator that asks "does this contain at least one letter?" says yes. Braille Blank is a symbol (So) — printable, so a "message must not be empty" check passes. Zero-Width Space is a format character (Cf), and format characters are the first thing a sanitiser strips, which is why it fails in WhatsApp and works in a text editor.

If you want to see the categories yourself:

for (const ch of ['ㅤ', '⠀', '', '⁣', ' ']) {
  const cp = ch.codePointAt(0).toString(16).toUpperCase().padStart(4, '0');
  console.log(`U+${cp}`, /\p{L}/u.test(ch) ? 'letter' : /\p{S}/u.test(ch) ? 'symbol' : /\p{Cf}/u.test(ch) ? 'format' : /\p{Zs}/u.test(ch) ? 'space' : 'other');
}
// U+3164 letter · U+2800 symbol · U+200B format · U+2063 format · U+200A space

Why an invisible name breaks after an update

A game that once accepted U+3164 and now rejects it did not "patch invisible names". Somebody added a Unicode normalisation or a category filter to the name validator — usually to stop impersonation, because two names that look identical but differ by an invisible character are a moderation nightmare. The moment a validator runs NFKC normalisation, U+3164 becomes U+1160 (Hangul Jungseong Filler) or is dropped entirely, and U+FFA0 (the halfwidth form) folds into the same thing.

That is why any honest invisible-text tool offers several characters instead of one, and says which app each survives. It is also why "test on your own account before spending a rename card" is the only advice that stays true across updates.

Detecting it is trivial, which is the point

Invisible does not mean hidden. Paste the text into anything that counts characters and the count gives it away:

const INVISIBLE = new Set([0x200B, 0x200C, 0x200D, 0x2060, 0xFEFF, 0x2800, 0x3164, 0xFFA0, 0x2063, 0x200A, 0x00A0]);
const report = (s) => {
  let invisible = 0;
  for (const ch of s) if (INVISIBLE.has(ch.codePointAt(0))) invisible++;
  return { total: [...s].length, invisible };
};
report('abㅤ⠀'); // { total: 4, invisible: 2 }

Two details that bite: iterate with for…of, not .length — .length counts UTF-16 code units, so an emoji is 2 and a family emoji is 11, and your "invisible" count drifts. And normalise after counting if you must normalise at all; s.normalize('NFKC') removes the very characters you are trying to find.

The other kind: text that carries a message nobody can see

There is a second, unrelated trick that gets called "invisible text" too. Zero-width characters come in pairs that a program can tell apart — U+200B and U+200C — so a string of them can carry bits. Encode each byte of a message as eight zero-width characters, wrap the run in a visible carrier so the chat app accepts it, and the recipient pastes the "empty" message back into the same page to read it. It is steganography, not encryption: anyone with the decoder reads it. The ghost text page does exactly that, and the newer invisible text page is the plain-blank version — one tap copies the character an app keeps, and a checker box tells you whether what you pasted really contains one.

Both run in the browser. There is nothing to upload; the "tool" is a clipboard write of a string the page already holds.

Practical summary

Blank message in WhatsApp, Telegram, Discord: U+2800.
Blank name or bio line in Instagram, TikTok, Free Fire, PUBG: U+3164 (fallback U+FFA0).
Blank post on X or a hidden line in Notes: U+200B.
Facebook / Messenger: U+2063. Documents and email: U+200A.
Nothing works in a field that validates against [a-z0-9._] — Instagram and Discord usernames — and no tool can change that.

If you maintain a name field and want to stop this: normalise with NFKC, then reject anything that is empty after removing \p{Cf}, \p{Zs} and the two Hangul fillers. Say so in the error message, because "invalid name" sends people straight back to the tools above.

Unicode's bold alphabet has holes in it, and every text generator falls in

Mahmoud Ibrahim — Confileo — Fri, 28 Aug 2026 04:13:52 +0000

If you have ever built a "fancy text" generator — the things people paste into
Instagram bios and Discord names — you have written this function:

// Looks right. Is wrong.
const toBold = text => text.replace(/[A-Za-z]/g, ch => {
  const c = ch.charCodeAt(0);
  return c <= 90
    ? String.fromCodePoint(0x1D400 + (c - 65))   // A-Z
    : String.fromCodePoint(0x1D41A + (c - 97));  // a-z
});

Map A onto U+1D400, add the offset, done. It works. You test it with
"Hello World", get 𝐇𝐞𝐥𝐥𝐨 𝐖𝐨𝐫𝐥𝐝, and ship it.

Then someone types their name in cursive and gets ☐.

The block is not contiguous

The Mathematical Alphanumeric Symbols block, U+1D400–U+1D7FF, holds about a
dozen styled Latin alphabets: bold, italic, bold italic, script, bold script,
fraktur, double-struck, sans-serif, monospace, and the digit sets.

It reads like a clean set of 26-letter runs. It is not. When the block was
encoded, some of those letters already existed in the Letterlike Symbols
block, U+2100–U+214F, because mathematicians had been using them as named
constants for decades — ℋ for the Hamiltonian, ℒ for the Lagrangian,
ℝ for the reals. Unicode does not encode the same character twice. So rather
than duplicate them, it left those slots in the new block permanently
unassigned.

Five alphabets have gaps. Here is the complete set:

Style	Missing letters	Where Unicode actually put them
Script	B E F H I L M R, e g o	ℬ ℰ ℱ ℋ ℐ ℒ ℳ ℛ, ℯ ℊ ℴ
Fraktur	C H I R Z	ℭ ℌ ℑ ℜ ℨ
Double-struck	C H N P Q R Z	ℂ ℍ ℕ ℙ ℚ ℝ ℤ
Italic (serif)	h	ℎ (Planck's constant)

Every one of those is a hole your arithmetic falls into. 𝒜 + 1 is not ℬ, it
is U+1D49D, which is nothing at all. Your users see tofu, and only for certain
letters, which is why this survives testing: "Hello" breaks in script and
fraktur, "World" does not.

The fix is a lookup, not cleverness

There is no formula. The gaps are historical accident, so the only correct
implementation is an explicit exception table consulted before the arithmetic:

type Style = {
  upper?: number; lower?: number; digit?: number;
  holes?: Record<string, string>;
};

const HOLES_SCRIPT = {
  B: 'ℬ', E: 'ℰ', F: 'ℱ', H: 'ℋ', I: 'ℐ', L: 'ℒ', M: 'ℳ', R: 'ℛ',
  e: 'ℯ', g: 'ℊ', o: 'ℴ',
};

function styleChar(ch: string, s: Style): string {
  if (s.holes?.[ch]) return s.holes[ch];       // check the table FIRST
  const c = ch.charCodeAt(0);
  if (c >= 65 && c <= 90  && s.upper !== undefined) return String.fromCodePoint(s.upper + c - 65);
  if (c >= 97 && c <= 122 && s.lower !== undefined) return String.fromCodePoint(s.lower + c - 97);
  if (c >= 48 && c <= 57  && s.digit !== undefined) return String.fromCodePoint(s.digit + c - 48);
  return ch;                                    // pass everything else through
}

The test that catches it is a pangram, checked for the reserved codepoints
rather than eyeballed:

const RESERVED = new Set([
  0x1D455, 0x1D49D, 0x1D4A0, 0x1D4A1, 0x1D4A3, 0x1D4A4, 0x1D4A7, 0x1D4A8,
  0x1D4AD, 0x1D4BA, 0x1D4BC, 0x1D4C4, 0x1D506, 0x1D50B, 0x1D50C, 0x1D515,
  0x1D51D, 0x1D53A, 0x1D53F, 0x1D545, 0x1D547, 0x1D548, 0x1D549, 0x1D551,
]);

const out = apply('How vexingly quick daft zebras jump', style);
const bad = [...out].filter(c => RESERVED.has(c.codePointAt(0)));
console.assert(bad.length === 0, style.id, bad);

Eyeballing does not work, because ☐ is what a missing font looks like too.
Checking the codepoints tells you which of the two you have.

Three more traps in the same neighbourhood

Iterate codepoints, not code units. Every character you are producing is
astral-plane — above U+FFFF, so two UTF-16 code units. The moment your input
contains an emoji, or someone pastes your own output back in, for (let i = 0; i < text.length; i++) splits a surrogate pair and you emit garbage. for…of
and [...text] both iterate by codepoint. Use them.

Seed your zalgo. Glitch text works by stacking combining diacritics —
U+0300–U+036F — on each letter, and there is no limit on how many can stack.
The obvious implementation reaches for Math.random(). Do not: if the output is
rendered reactively, every keystroke re-randomises the whole string, so the
text visibly crawls while the user types and the thing they copy is not the
thing they were looking at when they decided they liked it. Derive the seed from
the input and run a tiny PRNG:

let state = seed | 0 || 1;
const next = () => {
  state ^= state << 13; state ^= state >>> 17; state ^= state << 5;
  return (state >>> 0) / 4294967296;
};

Make upside-down text an involution. Flipping is two operations: substitute
each character for its turned twin (a → ɐ, U+0250), then reverse the
string, because a genuinely rotated line reads from what was its end. Skip the
reversal and you get mirror writing, which is the most common bug in these
tools. But there is a subtler one: half your users arrive with text that is
already upside down and want it turned back. If your table only maps a → ɐ,
pasting ɐ back in leaves it untouched. Fill the reverse direction from the
forward table and the same function does both jobs:

for (const [k, v] of Object.entries({ ...FLIP })) {
  if (!(v in FLIP)) FLIP[v] = k;   // only fill gaps; hand-written pairs win
}

The part nobody tells users

Styled Unicode is worse for accessibility and for search, and it is worth saying
so in the UI rather than leaving people to find out.

A screen reader does not read 𝗯𝗼𝗹𝗱 as "bold". It reads it as "mathematical
bold small b, mathematical bold small o, mathematical bold small l,
mathematical bold small d" — because that is what those characters are named.
Search engines do not match 𝘀𝗵𝗼𝗲𝘀 to a search for shoes either.

That makes these styles genuinely good for a decorative username or one accent
word, and genuinely bad for a whole bio, a business name people need to find, or
anything that has to be read aloud. Most generators do not mention it. It costs
one sentence and it is the difference between a tool and a trap.

Working reference

Every finding here came out of building the text tools on
Confileo — 26 styles, the
full holes table, seeded glitch and a two-way flip, all client-side. The
upside-down one is the quickest
way to check the involution claim: paste its output back into its own box and
you should get your original line, character for character.

If you are building your own, the short version is: check the table before you
do the arithmetic, iterate by codepoint, seed anything random, and tell your
users what the output costs them.

The Arabic PDF bug was never in my code — it was the library version

Mahmoud Ibrahim — Confileo — Wed, 19 Aug 2026 00:31:04 +0000

In my last post I wrote about why Arabic comes out of PDF extraction in reverse word order. A commenter asked the obvious follow-up question, and it is the one I want to answer properly:

Did you end up running a full bidi pass, or approximating with run-level heuristics? That mixed-direction case is usually where the heuristics fall apart.

Neither. And that turned out to be the whole point.

What I thought the problem was

The naive mental model goes like this: the PDF stores glyphs in the order they were painted, painting for RTL text runs right-to-left, so extraction hands you a line that reads last-word-first. Therefore you must reconstruct logical order yourself — which means implementing the Unicode Bidirectional Algorithm (UAX #9) over the positioned glyphs.

That is a genuinely hard thing to do, for a reason the commenter named exactly: UAX #9 needs a base paragraph direction, and the PDF never stored one. You have to infer it from the script of the runs. Get it wrong on a line like المبلغ: 128 SAR and your "fix" flips the Latin and the digits too. You have turned one bug into two, and the second one only shows up on invoices and CVs — which is to say, on almost every document a user actually cares about.

I wrote a chunk of that logic. It half worked, which is worse than not working, because half-working code makes you write more of it.

What the problem actually was

Then I ran the same PDF through the same code on a different machine and got correct output.

The extraction library was handing me visual order on one machine and logical order on the other. Same file, same code, different dependency version.

LibreOffice. Version 7.1 (2021), which is what a lot of stable server distributions still ship, reverses Arabic on PDF import — it gives you paint order. A modern LibreOffice extracts logical order directly. The server was on 7.1. My laptop was not.

PyMuPDF. Same class of bug, different library. Below 1.24, Arabic comes back visual and reversed. From 1.24 onward it comes back logical.

So the correct amount of bidi reconstruction code is zero. Both libraries already do it, in the versions where they do it. Everything I had written was scrambling text that had already arrived in the right order, on exactly the machines where the library was correct — which is why it looked like an intermittent bug and cost me months.

The trap that hid it for weeks

Here is the part worth stealing, because it is what made this so hard to see.

The pipeline had two engines: LibreOffice as primary, and pdf2docx (which sits on PyMuPDF) as a fallback. Sensible design. Then PyMuPDF 1.26.5 removed Rect.get_area(), a method pdf2docx 0.5.8 still calls:

# pdf2docx 0.5.8, somewhere in layout analysis
if bbox.get_area() / page_area > threshold:

AttributeError. The fallback engine crashed on import-time-adjacent code paths, the orchestrator caught it, and everything silently fell through to the other engine.

Nothing was logged as broken because nothing was broken from the caller's point of view — output came back, tests passed, files converted. But I could no longer tell which engine had produced any given file, so I was debugging Arabic ordering against an engine that had not run in weeks.

The fix was a two-line shim rather than pinning PyMuPDF backwards:

def _ensure_pdf2docx_compat():
    """pdf2docx 0.5.8 calls Rect.get_area(), removed in PyMuPDF 1.26.
    Restore it rather than pinning back to a version that returns visual order."""
    import fitz
    for cls in (fitz.Rect, fitz.IRect):
        if not hasattr(cls, 'get_area'):
            cls.get_area = lambda self: abs(self.width * self.height)

The general lesson: a silent fallback is a debugging tarpit. If your orchestrator can switch engines without telling you, the first thing to add is not a retry, it is a log line naming which engine produced the output.

One infrastructure footgun on the way

Upgrading LibreOffice on a server usually means dropping a newer build somewhere and repointing the binary:

ln -sf /opt/libreoffice26.2/program/soffice /usr/bin/soffice

ln -sf will happily create that symlink when the target does not exist yet. You get a dangling link, LibreOffice stops working entirely, and the error you get back has nothing to do with symlinks. Confirm the binary is there before repointing, and restart the service afterwards — a long-running process that resolved soffice at startup will keep using the old path until it does.

What is left after you delete the reversal code

Not nothing, but much less than you would expect. Two things still need doing.

Normalise presentation forms. Some PDFs encode Arabic using the Unicode presentation-form blocks (U+FB50–FDFF, U+FE70–FEFF) — the pre-shaped initial/medial/final glyph variants — instead of base letters. The text is technically correct and will look fine, but it is not searchable, will not match a query typed normally, and behaves badly in Word. NFKC folds them back:

import unicodedata
text = unicodedata.normalize('NFKC', text)

Note that this is the only normalisation you want here. NFKC is safe for this because the presentation forms have canonical decompositions to their base letters. Do not go further and start stripping diacritics; you will damage Quranic text and vowelised teaching material.

Mark direction in the output format. For DOCX, logical order in the string is not enough — Word needs to be told the run is RTL, or it renders correctly-ordered text with left-aligned paragraph flow, which looks subtly wrong to a native reader:

<w:rPr>
  <w:rtl/>
  <w:rFonts w:cs="Arabic Typesetting"/>
</w:rPr>
<w:pPr>
  <w:bidi/>
</w:pPr>

w:bidi on the paragraph, w:rtl on the run, and a complex-script font via w:cs — miss the last one and Word substitutes something that breaks the cursive joins.

The takeaway

If you are debugging reversed RTL text in an extraction pipeline, check your library versions before you write a single line of bidi logic. The odds are high that the layer below you already solved it and you are about to un-solve it.

And add a version matrix to your i18n tests. Not a version floor — a matrix. This class of bug is invisible in a test suite that only ever runs one version of the thing doing the extracting, and it will reappear the day someone deploys to a distribution that pins an older package.

I write these up as I hit them while building Confileo, a free PDF toolkit where Arabic support is the point rather than an afterthought. If you have a PDF that still comes out wrong, I would genuinely like to see it — the interesting bugs are always in the files, not in the spec.

Why Arabic text comes out backwards when you extract it from a PDF (and how to fix it)

Mahmoud Ibrahim — Confileo — Sat, 04 Jul 2026 00:15:53 +0000

If you've ever built a feature that extracts text from PDFs, an Arabic-speaking user has probably filed this bug: "the words come out in reverse order." Not the letters — the words. Every line reads last-word-first.

I spent the better part of a year fixing this class of bugs while building Confileo, a free PDF toolkit with first-class Arabic support. Here's what's actually going on, because almost every explanation online is wrong or incomplete.

The four distinct failure modes

People say "Arabic breaks" as if it's one bug. It's four:

1. Visual vs logical order (the reversed-words bug)

A PDF doesn't store text the way a Word file does — it stores positioned glyph runs: "paint these shapes at these coordinates." For left-to-right scripts, the paint order happens to match the reading order, so naive extraction works by accident.

Arabic is right-to-left. Many PDF generators emit the glyph runs in visual order — the order they appear on screen, left to right. A naive extractor concatenates the runs as stored and produces every line word-reversed. The text was never "reversed" in the file; your extractor just assumed paint order == reading order.

Fix: reconstruct logical order using glyph positions + the Unicode Bidirectional Algorithm (UAX #9), not the content-stream order. Libraries like PyMuPDF already return text in logical order — a common mistake is "fixing" that output by reversing it again, which is how you get double-reversed text. Rule of thumb: never reverse Arabic yourself. If it looks backwards, your rendering layer lacks bidi support; the data is usually fine.

2. Disconnected letters (the ransom-note bug)

Arabic letters are contextual: ع renders differently in initial, medial, final and isolated positions, and letters join. That joining is applied at render time by a shaping engine (HarfBuzz being the standard).

If any step of your pipeline round-trips text through a non-shaping renderer — a canvas library, a barebones PDF writer, an image caption filter — you get isolated forms: م ر ح ب ا instead of مرحبا. (Fun fact: ffmpeg burns subtitles correctly only because libass links HarfBuzz + FriBidi. Strip those and Arabic subtitles shatter.)

Fix: only render Arabic through stacks that do real shaping. If you're generating PDFs headlessly, Chromium/Puppeteer shape correctly; many lightweight PDF writers do not.

3. Font substitution (the hollow-boxes bug)

Your server converts a document referencing a font it doesn't have. The substitution fallback has no Arabic glyphs → tofu boxes. This is purely an ops problem: conversion servers get provisioned with Latin fonts only.

Fix: install a proper Arabic set (Noto Naskh Arabic, Amiri, Cairo) on every box that renders documents, and configure fontconfig so substitution prefers them.

4. Scanned PDFs (the empty-output bug)

A scanned PDF has no text layer at all — each page is a bitmap. Extraction returns nothing, or worse, your pipeline silently emits an "editable" file full of images. The only real answer is OCR, and generic OCR models trained mostly on Latin do poorly on connected Arabic script; use one with genuine Arabic training (PaddleOCR and Tesseract's ara traineddata are workable starting points).

Numbers and mixed text: the boss level

Dates like 2024/05/13 inside an RTL sentence follow their own bidi embedding rules — digits are LTR within an RTL context. Get the algorithm wrong and "من 2020 إلى 2024" turns into "من 2024 إلى 2020", silently corrupting the meaning. This is why you implement UAX #9 (or use a library that does) instead of hand-rolling direction logic.

Try it / steal the approach

I've put all of this into production converters you can poke at for free — Arabic Word → PDF (shaping + bundled fonts) and Arabic PDF → Word (logical-order extraction). No signup needed, so they're easy to use as a reference for what correct output should look like when testing your own pipeline.

If you're dealing with the same class of bugs in Hebrew or Farsi — same four failure modes, same fixes. Questions welcome in the comments; I've probably hit whatever edge case you're stuck on. 🐪