Making Search Work for Arabic: Hamza, Taa Marbuta and Prefixes in Plain JavaScript

We run a library of 200 free browser tools: PDF, images, video, calculators, text and developer tools. Every tool has an English page and an Arabic page, and one search box has to find the right tool in either language.

English search is the easy half. Lowercase the query, split it into words, maybe strip a plural "s", and you are most of the way there.

Arabic breaks every one of those assumptions. This post walks through the search we ended up with: about 200 lines of plain JavaScript, no library, no server, no index file. Everything runs in the browser over the tool cards that are already on the page.

Why includes() fails for Arabic

Take the word for "invoice". All of these mean the same thing, and real users type all of them:

What people type	What changed
فاتورة	the dictionary form
فاتوره	taa marbuta (ة) typed as haa (ه), very common on phones
الفاتورة	the article "al-" glued to the front
بالفاتورة	"bi-al-" ("with the") glued to the front
بِالفَاتُورَة	full vowel marks, copied from a document

A plain name.includes(query) finds one of these five. The same problems hit hamza (أ إ آ ء ؤ ئ), alef maqsura (ى vs ي), Arabic-Indic digits (٢٠٢٦ vs 2026), the tatweel stretch character (ـ), and Persian letters that come from some keyboards (ی ک گ پ چ).

So the search works in three steps. It folds every string into one canonical spelling, generates the forms a word can take, and then scores every tool.

Step 1: fold the text

Here is one word going through the whole pipeline:

The folding function is short (shown with arrow functions for brevity):

var CHARS = { "ة": "ه", "ى": "ي", "ی": "ي", "ء": "", "ٱ": "ا", "ک": "ك", "گ": "ك",
  "پ": "ب", "چ": "ج", "ڤ": "ف", "ظ": "ض", "ذ": "ز", "ث": "س" };

// NFKD splits أ إ آ ؤ ئ and Latin accents into letter + mark;
// the marks are then dropped with the diacritics
function fold(s) {
  return s.normalize("NFKD").toLowerCase()
    .replace(/[\u0300-\u036f\u0610-\u061a\u064b-\u065f\u0670\u06d6-\u06ed\u0640]/g, "")
    .replace(/[\u0660-\u0669]/g, d => d.charCodeAt(0) - 0x660)   // Arabic-Indic digits to 0-9
    .replace(/[\u06f0-\u06f9]/g, d => d.charCodeAt(0) - 0x6f0)   // Persian digits to 0-9
    .replace(/[ةىیءٱکگپچڤظذث]/g, c => CHARS[c])
    .replace(/[^\p{L}\p{N}]+/gu, " ").trim();
}

A few things worth pointing out:

• normalize("NFKD") does the hamza work for free. أ decomposes into ا plus a combining hamza mark, and the next regex removes the mark. The same call turns é into e, so English accents are handled by the same line.
• The character map also folds letters people swap by sound. ظ and ض, ذ and ز, ث and س sound alike in many Gulf dialects, and people misspell them constantly. Folding them on both sides (the query and the tool names) means the misspelling still matches.
• Digits become Latin digits, so a search for a year works whichever keyboard typed it.
• Folding is applied to the data too. The tool names are folded once when the page loads, so the comparison is always canonical against canonical.

Step 2: generate the forms of a word

Arabic glues articles and prepositions onto the word. A real stemmer for Arabic works with three-letter roots and is a project of its own. For a search box over 200 short tool names, light stemming is enough: strip one known prefix and one known ending, and keep the original word as well.

var AR_PRE = ["وال", "بال", "فال", "كال", "لل", "ال"], AR_PRE1 = "وفبلك";
var AR_SUF = ["ات", "ون", "ين", "ان", "يه", "ها", "هم", "كم", "نا", "ه", "ي"];
var EN_SUF = ["ations", "ation", "ings", "ing", "ions", "ion", "ers", "er",
              "ors", "or", "es", "s", "ed", "e"];

The function forms(word) returns the word plus the versions without a prefix and without an ending. For بالفاتوره it returns six forms:

forms("بالفاتوره")
// ["بالفاتوره", "فاتوره", "الفاتوره", "بالفاتور", "فاتور", "الفاتور"]

Two rules keep this safe:

1. At least three letters must survive. Without that rule, a short word like بال would be stripped down to nothing.
2. The plain word is always kept. Stripping only adds candidates, so a wrong strip can never lose a match that the plain word would have found.

English goes through the same function with EN_SUF, which is why compressor, compress and compressing all land on the same tools.

Step 3: allow one typo

After folding and stripping, a typo is the last common reason for a miss. The search uses an edit distance that also counts a swap of two neighbouring letters as one edit (Damerau, the "optimal string alignment" variant), so mrege is one edit from merge, not two.

// edit distance with swaps of neighbours; stops early above max
function dist(a, b, max) {
  if (Math.abs(a.length - b.length) > max) return max + 1;
  var d = [], i, j;
  for (i = 0; i <= a.length; i++) d[i] = [i];
  for (j = 1; j <= b.length; j++) d[0][j] = j;
  for (i = 1; i <= a.length; i++) {
    var low = max + 1;
    for (j = 1; j <= b.length; j++) {
      var c = a[i - 1] === b[j - 1] ? 0 : 1;
      d[i][j] = Math.min(d[i - 1][j] + 1, d[i][j - 1] + 1, d[i - 1][j - 1] + c);
      if (i > 1 && j > 1 && a[i - 1] === b[j - 2] && a[i - 2] === b[j - 1])
        d[i][j] = Math.min(d[i][j], d[i - 2][j - 2] + 1);
      if (d[i][j] < low) low = d[i][j];
    }
    if (low > max) return max + 1;   // the whole row is already too far
  }
  return d[a.length][b.length];
}

The early exit matters. The search runs on every keystroke against every word of 200 tools, and most pairs are obviously different. The length check rejects many of them before the table is even built, and the row check stops the rest as soon as no path can come back under max.

Words of 4 to 7 letters get one typo, words of 8 or more get two, and words under 4 letters get none. Short words have too many one-edit neighbours.

Step 4: score, don't filter

Every tool gets a score for every word of the query. The best kind of match wins for that word, and the word scores are added:

The modifiers do most of the ranking work:

• Name words count double. A word in the tool's name beats the same word in its description.
• Coverage beats strength. Each query word that matches anything adds 10, so a tool that matches all three words always beats a tool that matches one word perfectly.
• Word order counts. If the folded words appear in the tool text in the order typed, the tool gets +5. That is the whole reason jpg to pdf and pdf to jpg return opposite tools first.
• Ties go to what people actually search for. Each tool carries its monthly search volume in Saudi Arabia, and that adds a tiny, log-scaled bonus. The shorter, more specific name wins after that. Both are kept under 0.2 together, below the smallest real gap between two scores, so they only ever break ties.

Two more rules handle the edges:

• Joined and split words. If the query with its spaces removed is found inside the tool text with spaces removed, it counts as a full match. That makes whats app find WhatsApp.
• Let one word miss. With three or more words, one may miss. If still no tool passes, the search allows one more miss instead of showing an empty page.

Here is the real page, recorded from the live site:

Step 5: the wrong keyboard layout

This one is specific to bilingual users, and it happens all the time. You switch windows, start typing calc, and the keyboard is still on Arabic. The search box receives ؤشمؤ.

When a query returns nothing, the search reads the same physical keys on the other layout and tries again:

// standard Arabic (101) keyboard: the key each Arabic letter sits on
var KEYS = "qضwصeثrقtفyغuعiهoخpح[ج]دaشsسdيfبgلhاjتkنlم;ك'طzئxءcؤvرnىmة,و.ز/ظ`ذ";
var TO_AR = { b: "لا" }, TO_EN = {};   // plus the shifted hamza keys, omitted here
for (var k = 0; k < KEYS.length; k += 2) {
  TO_AR[KEYS[k]] = KEYS[k + 1];
  TO_EN[KEYS[k + 1]] = KEYS[k];
}

It only runs when the first search finds nothing, and only when the query is all Arabic or all Latin, so it never hijacks a real result. When it is used, the page says so: Showing results for "calc".

The b key is a nice detail. On the Arabic layout it types two letters at once, لا (lam-alef), so the reverse mapping has to read two characters as one key.

What it still gets wrong

Light stemming has limits, and it is better to know them:

• Stacked prefixes. وبالفاتورة ("and with the invoice") has two prefixes. The search strips one, so it finds the bill-splitting tools, which contain الفاتورة, before the invoice generator. A second pass would fix this, at the cost of more false matches.
• No roots. Arabic words grow from three-letter roots, and a real morphological analyser would connect words our stripping never will. For 200 tool names, the extra weight is not worth it.
• Stop words are a fixed list. Words like "tool", "free", "online" and their Arabic equivalents are ignored unless they are the only words typed.

Takeaways for any non-English search

1. Fold both sides. Whatever you do to the query, do to the data, once, at load time.
2. Let Unicode do the work. normalize("NFKD") plus one regex removes accents and Arabic marks in one go.
3. Add candidates, never replace. Keep the original word next to every stripped form, and stripping can only help.
4. Rank instead of filtering. A sorted list with a forgiving fallback beats an empty "No results".
5. Watch real queries. The keyboard-layout fix and the sound-alike letters both came from what people actually typed, not from a linguistics book.

You can try all of this on the live page: webnaa.com/tools/en. There is no sign-up and there are no ads, and most tools process your files in the browser without uploading them.

We are Webnaa, a Saudi web design team based in Riyadh. We build websites and online stores for businesses in Saudi Arabia and the Gulf, and we made these tools for the same people. This post was written by Claude, an AI assistant, working from our live search code. Every code sample and result was checked against that code.