Javascript remove duplicate from array of objects

const seen = new Set();
const arr = [
  { id: 1, name: "test1" },
  { id: 2, name: "test2" },
  { id: 2, name: "test3" },
  { id: 3, name: "test4" },
  { id: 4, name: "test5" },
  { id: 5, name: "test6" },
  { id: 5, name: "test7" },
  { id: 6, name: "test8" }
];

const filteredArr = arr.filter(el => {
  const duplicate = seen.has(el.id);
  seen.add(el.id);
  return !duplicate;
});

Comments

[Larson]

const filteredArr = arr.filter(
    (filterItem, index, self) =>
        index === self.findIndex((findItem) => findItem.id === filterItem.id),
);

[Larson]

There is no reason to store the objects in a Set in the first place so that conversion is entirely unnecessary.

Just to see what happens.

My bars on Firefox are so randomly but I guess it maybe has to do with my MacBook and the M1 Chip. And Firefox is not native here.

MacBook Pro (13-inch, M1, 2020) (m1 chip)

We should also mention Safari here.
Safari Version 15.1 (17612.2.9.1.20) (native)

Its not random you can reproduce it everytime. The performance is not even close to my Chrome Version 96.0.4664.55 (arm64)

[Larson]

yes.
some more functions: LINK

in general a while loop is the fastest (but sometimes not, 98% of the time it will). its not recommended. Because filter is also using a while loop but also checks the values if undefined etc. It is safer to use filter. Or we need to check manually. Which would lead in a slower approach.

/* withLoop */
const found = [];
const newArray = [];

const withLoop = array => { 
  let i = 0;
  while(i < array.length) {
    if(!found.includes(array[i].id)) {
      found.push(array[i].id);
      newArray.push(array[i]);
    }
    ++i;
  }
  return newArray;
}

The whole Set loop method it also fast, but when you need to return an array instead its slows the function down.

[peerreynders]

in general a while loop is the fastest

Your sample code moves the temporary objects to the top level which exaggerates the difference.

but when you need to return an array instead its slows the function down.

There is no reason to store the objects in a Set in the first place so that conversion is entirely unnecessary.

One also has to keep in mind that V8 has a spread optimization which is going to penalize overuse of the spread syntax on other JavaScript engines.

Perflink

1. withFilter
2. withLoopSet
3. withLoopIndexOf
4. withLoop
5. withSet
6. withFor
7. control (withFor)

Results for Chromium 96 on Ubuntu 20.04

The shape of that chart changes wildly with each run. To some degree that is to be expected as JavaScript performance is notoriously unpredictable.

Judging from the results on Firefox (94.0)

combining filter with Set doesn't help that much away from V8.