How to remove duplicates from JavaScript array

Recently, I posted a small snippet on how to remove duplicates from an array using a filter function. That started a thread on all the different ways to achieve this. Therefore, in this post, I am covering different ways to solve this problem.
JavaScript logo

1. Creating new array and for loop

For loop and the new array would be the oldest possible solution. A solution you would use before array functions. Today you would do it this way.

const numbers = [1, 2, 3, 1, 2, 3, 1, 2, 3];

const newNumbers = [];

for(let i = 0; i < numbers.length; i++) {
  if(newNumbers.indexOf(numbers[i]) < 0)
    newNumbers.push(numbers[i])
}

console.log(newNumbers);

2. Filter function

JavaScript arrays contain many different built-in functions, and a filter is one of them. This function takes one parameter, the filter function, which then has three parameters. The first one is the element you are testing, the second is its index, and the last one is the original array.

const numbers = [1, 2, 3, 1, 2, 3, 1, 2, 3];

const newNumbers =
  numbers.filter(
    (element, index, array) =>
      array.indexOf(element) === index
    )

console.log(newNumbers);

3. Using the Set object

Using a set is one of the more modern solutions. Sets are part of the ES6 version of JavaScript, and they are objects that do not contain duplicates. You can create a set object using the original array and then convert it back to the array.

const numbers = [1, 2, 3, 1, 2, 3, 1, 2, 3];

const newNumbers = Array.from(new Set(numbers));

console.log(newNumbers);

These are the three most common ways to remove duplicates from an array. But programming is creative, and I am sure there are many more. If you know of any, please write it in the comments.

---

For more, you can follow me on Twitter, LinkedIn, GitHub, or Instagram.

Comments

[Chris Milson]

You could sort the array first and then all duplicates will be next to each other:

const numbers = [1, 2, 3, 1, 2, 3, 1, 2, 3];

const unique = [...numbers].sort().filter((v, i, arr) => i === 0 || arr[i - 1] !== v)

console.log(unique) // [1, 2, 3]

[Mahmoud Ibn Samy]

why did you use an instance of the numbers array, not the number array itself? " [...numbers].sort() not numbers.sort() "

[Kristijan Pajtasev]

That is a good question, sort is a destructive function. This means running sort on numbers array will change original numbers array.

[Mahmoud Ibn Samy]

wow I forgot that

[Mahmoud Ibn Samy]

Great ways but is it efficient to use the sets method? Especially in the big arrays.

[Andrew Bridge]

I'd imagine if you were using a Set from the start, and adding values via the .add method on the Set object, you wouldn't need to store or iterate through the all the duplicates. That won't help if you needed to retain all the dupe data for some other uses though.

[Mahmoud Ibn Samy]

I agree that in some use cases you can't use Set. Actually, in some cases, you can't even use it from the beginning, As an example, if you want to have a collection of unique objects, Set won't help.

[Kristijan Pajtasev]

There is a topic on the StackOverflow showing the set is faster. I do personally like filter more, but in most cases, you won't see the difference, and you can go for what looks best in your code.

[Mahmoud Ibn Samy]

That's good. Thanks.

[dan]

and,, you can create a funtion for it,,

const unique = arr => [...new Set(arr)];

console.log(unique(numbers))

[Kristijan Pajtasev]

Ah yes, spread operator, love that one :smilery: