## DEV Community is a community of 642,334 amazing developers

We're a place where coders share, stay up-to-date and grow their careers.

# Remove duplicates from an Array the short & sweet way!

Ajmal Hassan
Unapologetically curious

## tldr;

``````const numbers = [1,2,2,3,3,3,4,4,4,4]
console.log([...new Set(numbers)]) //output:  [1,2,3,4]
// OR
console.log(Array.from(new Set(numbers))) //output: [1,2,3,4]
``````

## The long version

lets start by looking at what a `Set` is...

According to MDN

Set objects are collections of values. You can iterate through the elements of a set in insertion order. `A value in the Set may only occur once;` it is unique in the Set's collection.

What we can understand from the above is that Sets are like Arrays, in the sense that they are a collection of values that you can iterate through but the prime difference lies in the fact that is highlighted above i.e

`A value in the Set may only occur once;`

The above property of a Set is what we will be leveraging to expell duplicates from our array.

#### Let's now cut to the chase and see some code,

so, consider an array riddled with duplicates

``````const numbers = [1,2,2,3,3,3,4,4,4,4]
``````

We're going to divide the process into two steps

### 1. Converting Array to Set

``````console.log(new Set(numbers)) //output: {1,2,3,4}
``````

As you may have noticed, The duplicates have been eliminated! also another thing to notice is that eventhough Sets and Arrays have their similarities, the notation they use are different.

### 2. Converting Set to Array

#### a) Using `spread operator`

``````console.log([...new Set(numbers)]) //output: [1,2,3,4]
``````

#### b) Using `Array.from()`

``````console.log(Array.from(new Set(numbers))) //output: [1,2,3,4]
``````

## Discussion (14)

Garet McKinley

I'd be curious to see performance comparisons for this vs traditional deduping, like Array.filter(). The two way conversion feels like an expensive operation, but I have no basis for that other than the feeling.

Ajmal Hassan

Infact, Array.filter() is more performant but it's a bit harder to understand it's working when compared to this.

``````array.filter((item, index)=> array.indexOf(item)===index)
``````

If performance isn't critical then the method in the post is more readable and simpler.

Leonardo Santos

I don't think the `Array.filter` + `Array.indexOf` approach is more performant. Unless I'm missing something.

Both are linear, so for every item you are filtering, you are also iterating over the array again to find an index. Therefore, in the worst case, it will end up having a quadratic `O(n^2)` performance, which is really bad.

Garet McKinley

Maybe it's just my own ignorance not being familiar with the Set syntax. But the Array.filter method is so widely used that I'd know instantly what that line did. The example in the original post would confuse me to no end if there were no comments. I'd definitely have to google to figure out what's going on and why.

I'm sure the syntax is nice when you're familiar with it, but it's just not something that's done very often, so I'd definitely always leave a comment before it if you're using it in a code base that other people will have to read eventually.

Ajmal Hassan

I agree, Set isn't something we commonly use.

Ankit kumar shukla

is this work with like array data [{id=1,name='a'},{id=1,name='b'},{id=2,name='c'},{id=3,name='d'},{id=2,name='e'},{id=3,name='f'},{id=4,name='ag'},{id=5,name='au'}]

Ajmal Hassan

No, it won't.

The above method only works for `Primitive Types (number, string, boolean, null, undefined)` in the array.

so,

``````const primitivesArr = [true, false, true, false, undefined, 1, undefined, 'hello', 1, 'hello']

console.log([...new Set(primitivesArr)])
// output: [true, false, undefined, 1, "hello"]
``````

For objects, you'll have to use `filter()`

Rohan Sawant

Bonus - This works similarly in Python too, you can convert a List to a Set and that removes the duplicates.

Ajmal Hassan

One thing though, Python has unordered sets so, it may not keep the order of the elements intact.

Ajmal Hassan • Edited

Yeah, because javascript uses Ordered Sets unlike say python.