DEV Community

Discussion on: Converting UTF-8 strings to ASCII using the ICU Transliterator

 
bartvanraaij profile image
Bart van Raaij • Edited

I've taken a look at Laravel's str_slug. It uses voku/helper/ASCII::to_ascii under the hood.
That lib and function uses a quite clever in-memory cache on runtime, in which every character is cached in an array:
github.com/voku/portable-ascii/blo...
So subsequent transforms are much faster because they don't need to be transformed again.
This is of course highly beneficial to the performance.

The output difference between my slugify() and voku's to_ascii is explained by the fact that the latter takes a locale into account (English by default).

That being said: my "bonus tip" slugify example was never meant to be production code. It's just another example of what the ICU Transliterator can do. Of course there are other libs out there that do the same kind of stuff, which are perhaps better/faster at doing so; because there's a lot of development in them.
I hope you liked my article anyway, even if it's not directly usable for you. 🤞🏻

Thread Thread
 
lito profile image
Lito

Oh! caches 😅

Your article is great! and is perfect as the subject say, to understand how UTF-8 and ASCII converion works.