Speed-up your internationalization calls up to 5-1000 times

Vincent Thibault — Thu, 28 Feb 2019 16:00:58 +0000

Context

It all started two years ago. I was working on a new PWA for a big social network written from scratch that needed a i18n module to handle different languages. The module had to:

handle interpolation.
handle PLURAL and SELECT expressions.
be lightweight (it’s a PWA, must run with limited bandwidth).
run fast (some users had low-end devices).

And that’s where things got creepy, the only possible library was Google Closure MessageFormat. It was not so fast on low-end devices and weighing heavily on our bundle. So I decided to write my own with performance in mind.

Fast forward to today, the problem is still the same with i18n libraries, so I opened-source 💋Frenchkiss.js a 1kb i18n library 5 to 1000 times faster than others.
Stay with me for a journey on performances optimizations.

👉 Time to speed up your webapp for mobile devices!

🤷 How are i18n modules working?

Under the hood, it sucks, some i18n modules are re-processing the translation on each and every calls, resulting in poor performances.

Here is an example of what can happen inside the translate function (really simplified/naive version of Polyglot.js).

const applyParams = (text, params = {}) => {
  // Apply plural if exists
  const list = text.split('||||');
  const pluralIndex = getPluralIndex(params.count);
  const output = list[pluralIndex] || list[0];

  // Replace interpolation
  return output.replace(/%\{\s*(\w+)\s*\}/g, ($0, $1) =>  params[$1] || '');
}

applyParams('Hello %{name} !', {
  name: 'John'
});
// => Hello John !

In short, on each translations call we split the text, calculate the plural index, create a RegExp and replace all occurrences by the specified given parameter if it exists and returns the result.

It's not that big of a deal, but are you fine doing it multiple time on each render/filter/directive call ?

👉 It's one of the first things we learn when building app in react, angular, vuejs or any other framework : avoid intensive operations inside render methods, filters and directives, it will kill your app !

Some i18n libraries are doing better !

Some others are optimizing things quite a bit, here comes Angular, VueJs-i18n, Google Closure for example.

How are they doing it ? Actually they parse the string only once and cache a list of opcodes to process them on the next calls.

If you aren’t familiar with opcodes, it’s basically a list of instructions to process, in this case just to build a translation. Here's a possible example of opcodes generated from translations :

[{
  "type": "text",
  "value": "Hello "
}, {
  "type": "variable",
  "value": "name"
}, {
  "type": "text",
  "value": " !"
}]

And how we print the result :

const printOpcode = opcodes => opcodes.map(code => (
  (code.type === 'text') ? code.value :
  (code.type === 'variable') ? (params[code.value] || '') :
  (code.type === 'select') ? printOpCode( // recursive
    params.data[params[code.value]] || params.data.other
  ) :
  (code.type === 'plural') ? printOpCode( // recursive
    params.list[getPluralIndex(params[code.value])] || params.list[0]
  ) :
  '' // TODO not supported ?
)).join('');

With this type of algorithm, more time is allocated for the first call that generate the opcode but we store it and re-use it for faster performance in the next calls :

It doesn't split the string.
It doesn't do intensive regex operation.
It just read the opcode and merge the result together.

Well, that rocks ! But is it possible to go further ?

🤔 How can we speed up things ?

💋Frenchkiss.js is going one step further, it compiles the translation into a native function, this one is so light and pure that the Javascript can easily JIT compile it.

How does it work ?

Quite simple, you can actually build a function from a string doing the following :

const sum = new Function('a', 'b', 'return a + b');

sum(5, 3);
// => 8

For further informations, take a look at Function Constructor (MDN).

The main logic is still to generate an opcode list but instead of using it to generate a translation we use it to generate an optimized function that will returns the translation without further process.

It’s actually possible because of the simple structure of interpolation and SELECT/PLUTAL expressions. It’s basically a returns with some ternary.

const opCodeToFunction = (opcodes) => {
  const output = opcodes.map(code => (
    (code.type === 'text') ? escapeText(code.value) :
    (code.type === 'variable') ? `params[${code.value}]` :
    (code.type === 'select') ? ... :
    (code.type === 'plural') ? ... :
    '' // TODO Something wrong happened (invalid opcode)
  ));

  // Fallback for empty string if no data;
  const result = output.join('+') || "";

  // Generate the function
  return new Function(
    'arg0',
    'arg1',
    `
    var params = arg0 || {};
    return ${result};
  `);
});

⚠️ Note: when building dynamic function, make sure to avoid XSS injection by escaping user input !

Without further ado, let's see the generated functions (note: the real generated functions are a little more complex, but you will get the idea).

Interpolation generated function

// "Hello {name} !"
function generated (params = {}) {
  return 'Hello ' + (params.name || '') + ' !';
}

By default, we still fallback to empty string to avoid printing "undefined" as plain text.

Select expression generated function

// "Check my {pet, select, cat{evil cat} dog{good boy} other{{pet}}} :D"
function generated (params = {}) {
  return 'Check my ' + (
    (params.pet == 'cat') ? 'evil cat' :
    (params.pet == 'dog') ? 'good boy' :
    (params.pet || '')
  ) + ' :D';
}

We don't use strict equality to keep supports for numbers.

Plural expression generated function

// "Here {N, plural, =0{nothing} few{few} other{some}} things !"
function generated (params = {}, plural) {
  const safePlural = plural ? { N: plural(params.N) } :{};

  return 'Here ' + (
    (params.N == '0') ? 'nothing' :
    (safePlural.N == 'few') ? 'few' :
    'some'
  ) + ' things !';
}

We cache the plural category to avoid re-fetching it in case of multiple checks.

🚀 Conclusion

With generated functions we were able to execute code from 5 to 1000 time faster than others, avoiding doing RegExp, split, map operations in rendering critical path and also avoiding Garbage Collector pauses.

Last best news, it's only 1kB GZIP size !

If you're searching for a i18n javascript library to accelerate your PWA, or your SSR, you should probably give 💋Frenchkiss.js a try !

DEV Community: Vincent Thibault