U+FEFF (Zero-width space) - Fighting an invisible bug.

Abolade Eniseyin — Sat, 30 Jan 2021 00:18:14 +0000

So this is my first article here (anywhere) but I'll be brief.

I started the day high on energy, geared up to once and for all finish this feature that has taken too long. Who would have guessed I would be where I started Nine hours after 🤧.

So what is FEFF?

The zero-width space (), abbreviated ZWSP, is a non-printing character used in computerised typesetting to indicate word boundaries to text processing systems when using scripts that do not use explicit spacing, or after characters (such as the slash) that are not followed by a visible space but after which there may nevertheless be a line break. It is also used with languages without visible space between words, for example Japanese. Normally, it is not a visible separation, but it may expand in passages that are fully justified.
-- Wikipedia

Here, a simple explanation of this character and a tool to remove ZWSP in strings.

The presence of zero width space (ZWSP) can interfere with URLs, usernames and passwords.

How did this affect my day?

Well, unbeknown to me, the endpoint (url) I copied and pasted into my code (yeah I know, bad practice) had the zero-width space included. It looked just like this services/providers but it was services/providers.
Of course no visual difference (copy and see).

On calling the endpoint I repeatedly got a 405 from the api server but the curl example in the documentation worked perfectly in the terminal 😱. So what went wrong?

POSTMAN to the rescue

I dumped & copied my generated url string and used the same in postman to hit the endpoint directly and the error remained.

.
.
.

After taking a rest, I remembered Postman Console and checked it, Alas! I discovered a strange addition to my url.

services%E2%80%8B/airtime%E2%80%8B/request?

🏃🏻‍♂️🏃🏻‍♂️🏃🏻‍♂️ Stack overflow and others

No mention of such issue with my web framework [Laravel], a quick look up of Utf-8 reference sheet

So we have a clue, we know what the issue is but how do we know what added zero-width space to the string. Another rest was in order as this was already driving me crazy.

Back again
@manojow on Laracast gave this answer to similar issue, this provided a way to solve the issue, but you know we still need to understand the source of the issue right?

This answer displayed above gave me another name to search for. U+FEFF.
So we know the character is coming from somewhere outside the framework, as no such bug was reported by the thousands using Laravel's http facade.

Off I went for launch at 5pm, coming back it occurred to me to that this must have been because I copied all the endpoints. So I decided to type it instead and voila! - It worked.

This was supposed to be simple but it ended up wasting a ton of my time today. Silly in the end 😪.

So lesson learned: Be careful what you copy

So we end this with the advice of Eumir Gaspar in his post about FEFF.

If you do ever encounter an error like that, be sure to look for hidden characters not shown by your editor.

Did I keep my promise to be brief?.
See you 🖖.

Before I forget, we have a frog in the cover image if you didn't notice 😀.

DEV Community: Abolade Eniseyin

U+FEFF (Zero-width space) - Fighting an invisible bug.

So what is FEFF?

How did this affect my day?

POSTMAN to the rescue