DEV Community

Speed-up your internationalization calls up to 5-1000 times

Vincent Thibault on February 28, 2019

Context It all started two years ago. I was working on a new PWA for a big social network written from scratch that needed a i18n module...

Read full post

Mihail Malo • Mar 1 '19

I assume it would be more code on the wire if you server-rendered these functions?
Have your considered rendering them inside a Service Worker and returning the .js file?

Vincent Thibault • Mar 1 '19

Interesting idea to do it on the server-side/service-worker/build-tool !

Yet I'm not sure there is much gain, the generated function weighs more than the string representation. And it will mean for every update of the library you'll have to re-generate all your functions in case of signature mismatch (and possible fixes on the generated function).

But yeah, it would also be possible with this method to remove the compiler from the code and gain some extra bytes :)

By the way, here is how to extract the function if needed :

import { locale, set, t, cache } from 'frenchkiss';

locale('en');
set('en', {
  test: 'Check my {pet, select, cat{evil cat} dog{good boy} other{{pet}}} :D',
});

t('test'); // generate it
console.log(cache.en.test.toString()); // extract it

// =>
// function anonymous(a,f
// /*``*/) {
// var p=a||{};return "Check my "+(p["pet"]=="cat"?"evil cat":p["pet"]=="dog"?"good // boy":(p["pet"]||(p["pet"]=="0"?0:"")))+" :D"
// }
// */

Mihail Malo • Mar 1 '19 • Edited

var p=a||{};return "Check my "+(p["pet"]=="cat"?"evil cat":p["pet"]=="dog"?"good boy":(p["pet"]||(p["pet"]=="0"?0:"")))+" :D"

With a little more work of the compiler, there are things that could make the function shorter:

var p=a||{},a=p.pet;return "Check my "+(a=="cat"?"evil cat":a=="dog"?"good boy":a||(a=="0"?0:""))+" :D"

Can you clarify to me what the (a||(a=="0"?0:"")) is doing?

Vincent Thibault • Mar 1 '19

Yeah some optimizations can definitively cleanup the generated function. I just wanted to avoid spending much time (and file size) just to prettify the output.

The var p=a||{}; for example can be removed in case of raw string (that's not actually the case).

About the (a||(a=="0"?0:"")), it's actually to avoid printing "undefined", "null" in a translation, but keep "0" working :

// 'test' : 'Value: {value} !'

t('test'); // 'Variable:  !'
t('test', { value: undefined ); // 'Value:  !'
t('test', { value: null ); // 'Value:  !'
t('test', { value: 'test' ); // 'Value: test !'
t('test', { value: 0 ); // 'Value: 0 !'

Mihail Malo • Mar 1 '19

I'm not well-versed in i18n, is that the expected behavior of pet, select, other{{pet}}? Empty string for entirely missing key?

Mihail Malo • Mar 1 '19 • Edited

Can I suggest this way of inlining selects or will it hit performance?

var a=p&&p.pet;return "Check my "+{cat:"evil cat",dog:"good boy"}[a]||a||(a=="0"?0:"")+" :D"

If you do manage to get rid of new Function, you could do things like this though (n being a helper shared by all functions for all locales):

const n=x=>typeof x==="string"?x:"",
g=(c,x)=>c[x]||n(a)
// Elsewhere...
const c={cat:"evil cat",dog:"good boy"},
out=({pet:a}={},f)=>`Check my ${g(c,a)} :D`

Vincent Thibault • Mar 1 '19

Depend of the i18n libs, some are printing undefined, some others are keeping "{variable}" in translation.

As for me, I think it's a better user experience to have an empty string than a variable name (else the website seems broken).

Mihail Malo • Mar 1 '19

Should probably report an error in prod(in addition to whatever behavior you suggest)/fail in dev?
Imo the key name is less bad than an empty string, I have been on broken sites/apps where I could complete my flow in part thanks to the variable names in the template. But that's a nitpick.

Vincent Thibault • Mar 1 '19

Ok, I note the suggestion, I'll try to implement it this week end, something like the onMissingKey is working :

frenchkiss.onMissingVariable((value, key, lang) => {
  // report it
  sendReport(`variable "${value}" in ${lang}->${key} is missing.`);

  // the value you want to use instead
  return `[${value}]`;
});

Vincent Thibault • Mar 1 '19 • Edited

Just saw your comment about SELECT optimization.

I already did some tests with it working using an object mapping to values, but it doesn’t work well with nested expressions.

With nested expressions, you can’t really pre-cache objects and it will execute all the possible branches code before doing the resolution leading to performance issue.

Mihail Malo • Mar 1 '19 • Edited

With nested expressions you can have functions for some branches and strings for some. :v
Don't know about performance, but it is usually very compact, and you just need one helper:

const e = (table, x, p) => {
  const val = table[p]
  return typeof val === "function" ? val(x) : val
}
const a = { a: "A", b: "chungus" }
const b = { a: "B", b: x => `<${e(a, x, x.a)}>` }
const c = { a: "Cfunction b(x) {", b: x => `[${e(b, x, x.b)}]` }
const rule = x => `Yo, ${e(c, x, x.c)}`

const prop = { a: "b", b: "b", c: "b" }
console.log(rule(prop))
prop.a = "a"
console.log(rule(prop))
prop.b = "a"
console.log(rule(prop))
prop.c = "a"
console.log(rule(prop))

Vincent Thibault • Mar 4 '19

Not really so compact if you transpile it to ES5.
Here is an example of a complete solution (if I don't miss a thing ?).

The translation :

Updated: {minutes, plural,
  easteregg {never}
  =0 {just now}
  =1 {one minute ago}
  other {
    {minutes} minutes ago by {gender, select,
      male {male}
      female {female}
      other {other}
    }
  }
}

The global functions

// Global SELECT
function getBranchData(
  branch,
  prop,
  params,
  getPluralCategory,
  onMissingVariable,
  key,
  language
) {
  var data = branch.hasOwnProperty(prop) ? branch[prop] : branch.other; // fallback to 'others'

  return typeof data === "function"
    ? data(params, getPluralCategory, onMissingVariable, key, language)
    : data;
}

// Global PLURAL
function getBranchPluralData(
  branch,
  prop,
  params,
  getPluralCategory,
  onMissingVariable,
  key,
  language,
) {
  var category = getPluralCategory && getPluralCategory(params[prop]);
  var data;

  if (branch.hasOwnProperty(prop)) {
    // direct assignment
    data = branch[prop];
  } else if (category && branch.hasOwnProperty(category)) {
    // category check (easter egg)
    data = branch[category];
  } else {
    // default to other
    data = branch.other;
  }

  return typeof data === "function"
    ? data(params, getPluralCategory, onMissingVariable, key, language)
    : data;
}

// Global Interpolation
function handleInterpolation(params, prop, onMissingVariable, key, language) {
  return !params.hasOwnProperty(prop)
    ? onMissingVariable(prop, key, language)
    : typeof params[prop] === "number"
    ? params[prop]
    : params[prop] || "";
}

The generated function demo:

function functionGenerator() {
  // Closure to avoid re-defining branches at each call
  var branchA = {
    0: "just now",
    1: "one minute ago",
    other: function(params, getPluralCategory, onMissingVariable, key, language) {
      return (
        handleInterpolation(params, "minutes", onMissingVariable, key, language) +
        " minutes ago by " +
        getBranchData(
          branchB,
          params.gender,
          params,
          getPluralCategory,
          onMissingVariable,
          key,
          language
        )
      );
    }
  };

  var branchB = {
    male: "male",
    female: "female",
    other: "other"
  };

  return function(params, getPluralCategory, onMissingVariable, key, language) {
    return (
      "Updated: " +
      getBranchPluralData(
        branchA,
        params.minutes,
        params,
        getPluralCategory,
        onMissingVariable,
        key,
        language
      )
    );
  };
}

var fn = functionGenerator();
fn({
  minutes: 5,
  gender: "male"
});

I'll probably do a branch to see if it's a good candidate.

Mihail Malo • Mar 4 '19

        params,
        getPluralCategory,
        onMissingVariable,
        key,
        language

should probably be an object, passed by reference instead of individually through arguments?

Mihail Malo • Mar 1 '19

It's more about the fact that the browser can store the parsed and even potentially optimized function in cache, not just the string form, when you go through normal pathways.
new Function() is rather esoteric, and means it will definitely do a parse per instantiation, as well as cause some deoptimization around the instantiation.
Furthermore, using the library as is requires 'unsafe-eval' CSP directive on your entire page, which you otherwise might be able to avoid.