Kyle Simpson proved I STILL don't know JavaScript (arrays)

If you've never read Kyle Simpson's fantastic series, You Don't know JS, I'd first encourage to stop reading this, go read that , and then come back. It might take about six weeks, but that's fine. I'll wait.

...
...
...

I see you came back. Great. Feel like you know JavaScript now? You sure? Go back and read it again. I'll wait. This post isn't going anywhere.

...
...
...

Feel like you really know JS now? Fantastic. Let's talk about that simplest of data structures, the array, and what I didn't know after reading that book series four times.

JavaScript, I thought I knew you...

First I want to point you to this twitter thread where, in the third tweet in the series, Kyle makes this statement:

just for clarity sake, this is what I find surprising...

Now, I looked at that image and thought, "this makes sense. But... why does it make sense?"

The more I tried to think about why it made sense, the more I realized I didn't know JavaScript.

First, a primer on making arrays

There's a few different ways to make arrays in JavaScript. They amount to, "you can instantiate them with data, or without data".

Instantiating with data

Sometimes we need to write an array with data already in it. It's not uncommon to do this when you don't expect that list to really change after you declare it. So there's two ways we instantiate that array:

Using the Array constructor

const mistakes = new Array('my liberal arts degree', 'eating 15 slim jims')`

This is considered weird and generally a bad practice in JavaScript. I'll explain why in a bit.

Using the Array Literal

const moreMistakes = [tequila', 'Michael Bay\'s TMNT movie'];

This is the more common approach that I hope most of us are using at home.

Instantiating without data

In cases where we're moving data from one data structure to another we often declare an empty array and then modify it.

An extremely common pattern is declaring the empty array and then pushing to it:

const moreFutureMistakes = [];

moreFutureMistakes.push('TMNT sequels')

But if you want to be that person, of course you can use the array constructor:

const moreUnusualMistakes = new Array();

moreUnusualMistakes.push('what you\'re going to see next');

Weird ways to instantiate arrays

I don't think I've ever really seen these in the wild, but they've always been there in the back of my mind. Kind of like the Hawaii 5-0 theme song. It's not really doing anything in my brain other than sitting there. Making sure I don't forget it.

One of the only things I remember from my C++ class was that arrays had to have sizes. And I didn't know why. I still don't know why. (answer:somethingSomething[memory])

So three weird ways of instantiating arrays involve setting the size of it up front:

const superSizeMe = [];
superSizeMe.length = 3; // SURPRISE! Length is a setter

const preSized = new Array(3); // "This won't confuse anyone," said no one ever.

const commaSized= [,,,]; 

const isWeirdButTrue= (superSizeMe.length === preSized.length === commaSized.length);

If you were wondering why it's considered bad practice to use the Array constructor, now you know. It's because if you give it exactly one argument, and that argument is an Integer, it will create an array of that size. So the array constructor could get unexpected results when dealing with numbers.

And it's not like any of the other approaches are remotely close to best-practice, either. They're all the strange practices of someone who's maybe a little too curious for their own good or perhaps wondering if the trickster-god Loki is in fact alive and designing programming languages.

Weird Instantiations and Weird ways to set data result in weird results totally expected behavior.

Now we're caught up and back to Kyle's tweet about how weird this is:

[,,,3,4,,5].forEach(x=>console.log(x));
// 3
// 4
// 5

1. Ok, let's agree that comma-instantiated arrays is weird.
2. Ok. It logged... 3,4,5

This is fine. Everything's fine. those other slots must be undefined or unavailable.

for (let x of [,,,3,4,,5]) { console.log(x); }
// undefined 
// undefined
// undefined
// 3
// 4
// undefined
// 5

Hold up....

What's in those "weirdly instantiated arrays" ?

Let's take a step back and look at these pre-sized arrays:

const myRegrets = new Array(3);
const moreRegrets = [,,,];
const noRegerts = [];

noRegerts.length = 3;

If you're using Firefox, crack open the console, run this, and take a look at those arrays.

You might see something like:

Array(3) [undefined, undefined, undefined]

But is that array really filled with three undefined?

No. Not it is not. That's what Kyle Simpson is aptly pointing out. If you loop through these "pre-sized" arrays, and try to log the value, you will not get a log:

const myRegrets = new Array(3);
myRegrets.forEach((regret, regretIndex) => {
  console.log(regret, regretIndex);
});


for (regretName in myRegrets) {
 console.log(regretName, myRegrets[regretName]);
}

So, the first big takeaway here is that a pre-sized array, or an array created with comma/slots, doesn't have a value in those slots.

myRegrets is not an array with 3 undefined. It's an array with three slots of nothing.

To further prove this point, add an actual undefined to the third slot:

const myRegrets = new Array(3);
myRegrets[1] = undefined; 
myRegrets.forEach((regret, regretIndex) => {
  console.log(regret, regretIndex);
});


for (regretName in myRegrets) {
 console.log(regretName, myRegrets[regretName]);
}

You got a log, didn't you? Just one, right?

Double-you Tee Eff

There's an implicit and explicit undefined in Arrays

This is what I think Kyle is talking about here.

When we do these weird Array tricks where we pre-size it, or we comma-slot it (e.g. [,,undefined]), JavaScript isn't actually putting values in those slots. Instead, it's saying the slots exist ... kinda.

If something exists, but has no value, we have a name for that:

undefined

const myRegrets = [,,undefined];
const youWillRegretThis;

myRegrets[0] === youWillRegretThis; // true, so very true

But I call this "implicit undefined" because this doesn't get logged in any looping that we do. Not forEach, neither for - in, nor map and its buddies will log a slot that has no value; i.e. implicitly undefined;

You could also call this "undeclared" if you don't like "implicitly undefined".

Explicit undefined must take up memory

When you loop over an array with an explicit undefined , it must be taking up actual memory. And that's why it gets logged:

const myRegrets = [,,undefined];

myRegrets.forEach((regret, regretIndex) => {
  console.log(regret, regretIndex);
});
// will log 3

for (regretName in myRegrets) {
 console.log(regretName, myRegrets[regretName]);
}

// will log 3

So just remember this, kids. There's an implicit undefined and an explicit undefined when it comes to arrays.

This probably won't be a thing you trip on any time soon unless you merge arrays.

Or use for of...

Wait. for of?

yes.

(Double-you tee ay eff)

for - of is the only looping mechanism that doesn't care about implicit or explicit undefined.

Again, think of "implicit undefined" as meaning, "undeclared":

const myRegrets = [,,undefined];


let regretCounter = 0;

for (regret of myRegrets) {
 console.log(regret, regretCounter++)
}
// undefined 0
// undefined 1
// undefined 2

Why did it log all three?

I don't know for certain, but I have a theory that a much smarter person than I needs to investigate.

The for of pattern implements an iteration protocol

My theory here is that the iteration protocol on arrays behaves something like what's shown from the examples page:

function makeIterator(array) {
    var nextIndex = 0;

    return {
       next: function() {
           return nextIndex < array.length ?
               {value: array[nextIndex++], done: false} :
               {done: true};
       }
    };
}

if for - of implements something like this under the hood, this will iterate by index, not property (and arrays are really just objects with properties that are numbers. kinda).

So, back to what we call a thing that exists but has no value. You know the one. Our old friend. The one we never invite to parties but he shows up anyway? Remember him?

undefined

I'm really starting to not like that guy. He weirds me out.

TL;DR

1. The .length property of an array isn't the same as saying, "these many slots have value"
2. Arrays can have "implicit undefined", and "explicit undefined", depending on whether the space really has a value and "implicit undefined" is more like, "undeclared"
3. If you don't create arrays in weird ways, or do weird things with arrays, you'll likely never encounter this
4. If you are that trickster god Loki, or a fan, and choose to create arrays in weird ways, or manipulate them weirdly, you may need to use for of to get the most consistent result while looping.

JavaScript makes a lot of sense early in the morning. Not late at night.

[Edit: Some additions]

A few comments on this post made me do some research and testing that some may find useful. This was already a long read, so unless you're camped out in a bathroom stall on account of a late-night binge on cheese, or you're otherwise really interested in talking about specifications, you can skip this stuff.

Are there array properties that are simply being set as not enumerable?

I don't think so.

I skimmed through some ECMA specs to see if this behavior is defined anywhere.

The specs say that array elements without assignment expressions are not defined

Section 12.2.5 says

Whenever a comma in the element list is not preceded by an AssignmentExpression (i.e., a comma at the beginning or after another comma), the missing array element contributes to the length of the Array and increases the index of subsequent elements. Elided array elements are not defined.

So if you have [,,'foo'], those array elements after the comma that don't have some sort of expression are "elided".

Also worth noting is that the specs say that ['foo',] does not contribute to the length of the array.

Also also worth noting is that I haven't yet found if pushing values to a random index above the length counts as elision. e.g.:

const gapped = [];
gapped[2] = "please mind the gap";

The specs don't seem to state that array elements are created but not enumerable

Step 8 of section 22.1.1.3 describes how an array is created:

Repeat, while k < numberOfArgs
a. Let Pk be ! ToString(k)
b. Let itemK be items[k]
c. Let defineStatus be CreateDataProperty(array, Pk, itemK)
d. Assert: defineStatus is true
e. Increase k by 1

Pk is the key (which is an index) and itemK is the value.

If the JavaScript engine is following this algorithm, an item, regardless of its value, gets passed into the CreateDataProperty function/method/whatever.

The question is, "does the first slot in [,,'foo'] constitute being an item? 12.2.5 says no. (I think)

But is there a chance CreateDataProperty is creating a property making it non-enumerable?

If you read in section 7.3.4, it doesn't give any logic or condition where the enumerable property in the descriptor . Steps 3 and 4 set the property to be enumerable:

1. Let newDesc be the PropertyDescriptor { [[Value]]: V, [[Writable]]: true, [[Enumerable]]: true, [[Configurable]]:true }.
2. Return ? O.[DefineOwnProperty]

I haven't read through all the specification has to say about arrays. But this seems further suggest that these "empty slots" are really empty.

What about the in operator. Will it find these empty slots?

No it will not!

const slotted = [,,'one'];
let i = 0;

while (i < slotted.length) {
 if (i++ in slotted) {
  console.log(`${i - 1} is in the array`);
 }
}

This will log exactly once, displaying 2 is in the array in your console.

If you have an explicit undefined, howeverconst explicit = [undefined, undefined, 'one'] , it will log three times.

STL;SDR (Still too long, still didn't read)

First, let me caveat everything by telling you I'm way more qualified to talk to you about French Existentialism than JavaScript. The likelihood that all this is wrong is pretty high.

Based on my understanding of the specifications, "implicit undefined" is a somewhat valid-ish way to describe a "slot" in an array that has no value.

Except, of course, there's not really even a slot. There's not even the idea of a slot. A slot does not exist without a value. (#Existentialism)

As Kyle Simpson points out there is a difference between undefined and undeclared but JavaScript doesn't always give you messaging that makes it clear which is which.

This "implicit undefined" is more of an existential problem where we only have so many ways to describe Being and Nothingness.

const existentialCrisis= [,undefined,'Waiting for Godot']`;
console.log(typeof existentialCrisis[1]); // undefined
console.log(typeof existentialCrisis[0]); // undefined

Comments

[Ben Calder]

I generally don't concern myself too much with what's happening under the hood; but this was interesting :)

So - the output of this is enlightening:

const array1 = [, 'b', 'c'];
console.log(typeof array1[0]); // undefined
console.log(array1.propertyIsEnumerable(0)); // false

const array2 = [undefined, 'b', 'c'];
console.log(typeof array2[0]); // undefined
console.log(array2.propertyIsEnumerable(0)); // true

const array3 = new Array(3);
console.log(typeof array3[0]); // undefined
console.log(array3.propertyIsEnumerable(0)); // false

The implicit/explicit undefined looks like a red herring to me.

Array properties that haven't been explicitly declared are simply not set as enumerable; but I guess as far as the interpreter is concerned the values to be iterated over have been defined:

The for...of statement iterates over values that the iterable object defines to be iterated over. (MDN)

Also, see the output from console.log(array1.entries().next()).

[Matt McMahon]

Yep. Beat me to it. It's all the difference between having an ownProperty or not.

I'm wondering, since I've never tried to do this and can't conveniently try it right now, what the in operator reports for things like 0 in [,1,2]? Is that statement true or false? Or is it a syntax error?

[Paceaux]

I tested five loops: for in, for of, forEach, map and for of was the only one that iterated on all slots.

Here's a gist of what I wrote. Feel free to run it in a few different browsers and tell me if my tests were wrong.

[Paceaux]

I went ahead and updated my test gist with an if in test to see what it did.

const slotted = [,,'one'];
let i = 0;

while (i < slotted.length) {
 if (i++ in slotted) {
  console.log(`${i - 1} is in the array`);
 }
}

it, too, will inform us that only an index of 2 exists. This I think further confirms that these items in the array are not "hidden properties" at all.

[Paceaux]

You make a great argument, but I don't think it's a red herring. (I'm more than willing to be proven wrong, though).

I read your comment a few times and wondered, "If a property isn't explicitly declared, does that mean it's implicitly undefined?" i.e.... are we really saying the same thing?

I decided to dig into the specs to see if a property really is created, though. And I don't think one is.

I've found at least one spot in the ecma specs that defines this behavior in section 12.2.5:

Elided array elements are not defined.

When you say

Array properties that haven't been explicitly declared are simply not set as enumerable;

At least in this one section, it doesn't appear as though these empty slots are properties created on the array.

Most telling is step 8 of 22.1.3 of the specs that describe how an array is created:

Repeat, while k < numberOfArgsa.

a. Let Pk be ! ToString(k)
b. Let itemK be items[k
c. Let defineStatus be CreateDataProperty(array, Pk, itemK)\
d. Assert: defineStatus is true
e. Increase k by 1

So, Pk is the key, and itemK is the value. If you have an array containing an undefined, e.g. [undefined, 1, 2], it's an item ... at least I think ... according this logic.

Similarly, that would mean that in the case of [,1,2], there is no "item" in the first slot.

But that all depends on what CreateDataProperty does. When I read how CreateDataProperty behaves, in section 7.3.4, steps 3 and 4 don't give a condition for setting the enumerable prop in the descriptor to false:

1. Let newDesc be the PropertyDescriptor { [[Value]]: V, [[Writable]]: true, [[Enumerable]]: true, [[Configurable]]:true }.
2. Return ? O.[DefineOwnProperty]

So, if there were a value... any value, it would be enumerable. But if there isn't a value... it's not.

So I think "implicit undefined" is an accurate description of this case because the specifications dictate in at least one place that "elided array elements are not defined" ... meaning no properties are created on the array and browser implementers have to give you something if you ask for a value at a slot that has no value. So the Array internals are giving you undefined.

I think.

[Ben Calder]

Like I said - this isn't something I tend to dig into too much...

But I think we're more or less in agreement - I'm just not comfortable with the idea of a distinction between explicit/implicit undefined. However you get to it undefined === undefined :D

I didn't dig into the spec but did think to do this:

var array1 = [,"a","b"];
// since an Array is just a fancy Object:
console.log(Object.keys(array1)); // ["1", "2"]
console.log(array1.hasOwnProperty(0)); // false
console.log(array1.hasOwnProperty(1)); // true
console.log(array1.hasOwnProperty(2)); // true

// seems obvious but for ... of isn't the only way to 
// get the implicit undefined
for(let i = 0; i<array1.length; i++) {
  console.log(array1[i]);
} // undefined a b

array1[0] = "c";
console.log(Object.keys(array1)); // ["0", "1", "2"]

// and just for good measure:
console.log(Object.keys([,"a",,,"b",])); // ["1", "4"]

So the above suggests that:

• keys are only set for values that are explicitly declared
• modern iterator functions loop over Object.keys - i.e. enumerable properties
• for ... of assumes that Object.keys should conform to array standards (i.e. start at 0 increment by 1)
• the old fashioned for loop just does what you tell it :)
• undefined is always returned when you try to access an object property that doesn't exist

Thanks for the article! It turns out that thinking about what happens under the hood can consume far too much of my time :D

[Paceaux]

I updated my article to reflect our conversation (seemed too useful not to)

I think this goes back to an odder quirk of JavaScript where sometimes undefined also means "undeclared", but not always.

this "implicit undefined" is really more like an "undeclared", but JavaScript doesn't provide a means to distinguish the two very easily.

[Ben Calder]

Oh - and a massive +1 for recommending the "I don't know JS" series!

[Genspirit]

I think this actually makes perfect sense if you actually take a look at the
documentation. When you set something to undefined you are initializing it. So while it is still undefined its not the same as uninitialized. Array.forEach isn't invoked on uninitialized values and for... in(which you shouldn't use with arrays anyways) does something similar(more specifically it will loop over enumerable properties). What you are referring to as implicitly undefined and explicitly undefined is largely just initialized and not initialized.

[Paceaux]

You are absolutely right on all points (and that's really what I was getting to).

The behavior makes sense once you break down what's actually going on (as is almost always the case with JavaScript)

And yeas, "implicitly undefined" and "explicitly undefined" could also be called "uninitialized" and "initialized". Though I like implicit/explicit because it suggests intent a bit more clearly.

[nathan amick]

Wow, great write up. Made me actually laugh out loud.

If I didn't already love JavaScript, I would steer clear. It sounds like a language somebody conceived of and implemented in like ten days with no forethought or planning.

Reminds me how relevant Douglas Crockford's "JavaScript, The Good Parts" still is.

[Paceaux]

Thank you!

I think JavaScript is brilliantly well done. Regardless of the whole, "10 days and no forethought", I think Brendan Eich did a lot right.

I think he understood way better than many other programmers how important it is in the browser to have a fault tolerant language that does as much as possible to prevent the UI from crashing on a user.

I think most of the "quirks" or "weirdnesses" in JavaScript can be explained as being quite intentional because Eich was trying to make a language that wouldn't punish the end user for the developer's mistakes.

Except Date.

and NaN !== NaN.

Those I think he screwed up on.

[nathan amick]

Ha, indeed. I think you're right.

[Steffen Nilsen]

I haven't done any testing, but the part where you go over array instantiation and c++, I wonder how smart the V8 engine is when it comes to allocate memory for an array. I know that you shouldn't use sparse arrays, inserting items at indexes so there's a gap, but I can't recall if it does optimizations based on setting a predefined array size

[Paceaux]

If I were smart enough to work on the V8 engine, I'd have it optimized based on array size.

Granted, I only ever took one C++ class... and I have liberal arts degrees. So I'm far from qualified-enough to say for sure if that's the right way to go.

[Budy]

whooaa... dang you Array 😂😂