Removing Duplicate Objects from an Array of Objects

Arrays and Objects are two basic data structure we find ourselves using in our everyday JavaScript coding task. Though they are quite simple to use, as the number of data we use them to store grow very large, we find a need to optimize the time and resources taken to perform operations on them.

In this article, I will focus on two fast approach to remove duplicate objects from an array of objects.

Recently, while working on a project, I came across a task that requires to remove duplicates from an array of objects.

For example, say you have this array of objects and you are tasked to find and remove any duplicates in the array

const data = [
  {
    id: 1,
    title: "Pick meals",
    description: "Choose your meal"
  },
  {
    id: 2,
    title: "Checkout",
    description: "Enter a delivery location"
  },
  {
    id: 3,
    title: "Fast Delivery",
    description: "Your order is processed and delivered"
  },
  {
    id: 2,
    title: "Checkout",
    description: "Enter a delivery location"
  },
 {
    id: 1,
    title: "Pick meals",
    description: "Choose your meal"
  }
]

This task may look trivial at first but as you dive in to code and having optimization in mind, you will find it not as trivial as you thought. Below are two methods different methods to solve this task

Method 1 : Using Set

let data = [ { id: 1, title: "Pick meals", description: "Choose your meal" }, { id: 2, title: "Checkout", description: "Enter a delivery location" }, { id: 3, title: "Fast Delivery", description: "Your order is processed and delivered" }, { id: 2, title: "Checkout", description: "Enter a delivery location" }, { id: 1, title: "Pick meals", description: "Choose your meal" } ] //1st convert all object in the array to string to make it easy to compare them let result = data.map(item=>JSON.stringify(item)); //pass the array into a Set to remove duplicates result= new Set(result); // convert the Set back to array of strings result = Array.from(result); // finally convert the array of strings to an array of object result = result.map(item=>JSON.parse(item)); console.log(result);

It is important to stringify the objects first before passing them to the Set object else the Set will compare them by reference instead of by value i.e Since the object are store in different memory location, the Set will see them as different object even though they have the same exact values.

Method 2 : Using Objects

let data = [ { id: 1, title: "Pick meals", description: "Choose your meal" }, { id: 2, title: "Checkout", description: "Enter a delivery location" }, { id: 3, title: "Fast Delivery", description: "Your order is processed and delivered" }, { id: 2, title: "Checkout", description: "Enter a delivery location" }, { id: 1, title: "Pick meals", description: "Choose your meal" } ] //1st create an empty object let result = {}; //loop through the data array and use one of the keys of objects in the array to create properties in the result object data.forEach(item=>result[item.id] = item) // extract back the array from the object result= Object.values(result); console.log(result)

Both methods works well so far that the duplicate objects are exactly the same. Method 2 apart from being shorter executes faster.

Note

In some rare cases, if the definition of duplicate is altered for example given the
below data and

const data = [
  {
    id: 1,
    title: "Pick meals",
    description: "Choose your meal"
  },
  {
    id: 2,
    title: "Hey",
    description: "Enter a delivery location"
  },
  {
    id: 3,
    title: "Fast Delivery",
    description: "Your order is processed and delivered"
  },
  {
    id: 2,
    title: "Checkout",
    description: "Enter a delivery location"
  }
]

Method 2 will not be the best if duplicate means itemA is exactly the same as ItemB and some object has the same id but have some other properties different, in this case Method 1 is best to use. Similarly if duplicate means itemA has exactly the same 'id' as ItemB regardless of other properties then Method 2 is the best for to use.

I hope you find this article useful, if you like the content, feel free to stay in touch, follow me on twitter@_DVERYBEST.