How to find duplicate values in a JavaScript array of objects, and output only unique values?

Question

I'm learning JS. Supposing I have the below array of objects:

var family = [
  {
    name: "Mike",
    age: 10
  },
  {
    name: "Matt"
    age: 13
  },
  {
    name: "Nancy",
    age: 15
  },
  {
    name: "Adam",
    age: 22
  },
  {
    name: "Jenny",
    age: 85
  },
  {
    name: "Nancy",
    age: 2
  },
  {
    name: "Carl",
    age: 40
  }
];

Notice that Nancy is showing up twice (changing only the age). Supposing I want to output only unique names. How do I output the above array of objects, without duplicates? ES6 answers more than welcome.

Related (couldn't find a good way for usage on objects):

• Remove Duplicates from JavaScript Array
• Easiest way to find duplicate values in a JavaScript array

EDIT Here's what I tried. It works well with strings but I can't figure how to make it work with objects:

family.reduce((a, b) => {
  if (a.indexOf(b) < 0 ) {
    a.push(b);
  }
  return a;
},[]);

Answer 1

You could use a Set in combination with Array#map and a spread operator ... in a single line.

Map returns an array with all names, which are going into the set initializer and then all values of the set are returned in an array.

var family = [{ name: "Mike", age: 10 }, { name: "Matt", age: 13 }, { name: "Nancy", age: 15 }, { name: "Adam", age: 22 }, { name: "Jenny", age: 85 }, { name: "Nancy", age: 2 }, { name: "Carl", age: 40 }],
    unique = [...new Set(family.map(a => a.name))];

console.log(unique);

For filtering and return only unique names, you can use Array#filter with Set.

var family = [{ name: "Mike", age: 10 }, { name: "Matt", age: 13 }, { name: "Nancy", age: 15 }, { name: "Adam", age: 22 }, { name: "Jenny", age: 85 }, { name: "Nancy", age: 2 }, { name: "Carl", age: 40 }],
    unique = family.filter((set => f => !set.has(f.name) && set.add(f.name))(new Set));

console.log(unique);

Answer 2

The Solution

Store occurrences of name external to the loop in an object, and filter if there's been a previous occurrence.

https://jsfiddle.net/nputptbb/2/

var occurrences = {}

var filteredFamily = family.filter(function(x) {
  if (occurrences[x.name]) {
    return false;
  }
  occurrences[x.name] = true;
  return true;
})

you can also generalize this solution to a function

function filterByProperty(array, propertyName) {
  var occurrences = {}

  return array.filter(function(x) {
    var property = x[propertyName]
    if (occurrences[property]) {
      return false;
    }
    occurrences[property]] = true;
    return true;
  })
}

and use it like

var filteredFamily = filterByProperty(family, 'name')

Explanation

Don't compare objects using indexOf, which only uses the === operator between objects. The reason why your current answer doesn't work is because === in JS does not compare the objects deeply, but instead compares the references. What I mean by that you can see in the following code:

var a = { x: 1 }
var b = { x: 1 }

console.log(a === b) // false
console.log(a === a) // true

Equality will tell you if you found the same exact object, but not if you found an object with the same contents.

In this case, you can compare your object on name since it should be a unique key. So obj.name === obj.name instead of obj === obj. Moreover another problem with your code that affects its runtime and not its function is that you use an indexOf inside of your reduce. indexOf is O(n), which makes the complexity of your algorithm O(n^2). Thus, it's better to use an object, which has O(1) lookup.

Answer 3

This will work fine.

const result = [1, 2, 2, 3, 3, 3, 3].reduce((x, y) => x.includes(y) ? x : [...x, y], []);

console.log(result);

Answer 4

With the code you mentioned, you can try:

family.filter((item, index, array) => {
  return array.map((mapItem) => mapItem['name']).indexOf(item['name']) === index
})

Or you can have a generic function to make it work for other array of objects as well:

function printUniqueResults (arrayOfObj, key) {
  return arrayOfObj.filter((item, index, array) => {
    return array.map((mapItem) => mapItem[key]).indexOf(item[key]) === index
  })
}

and then just use printUniqueResults(family, 'name')

(FIDDLE)

Answer 5

I just thought of 2 simple ways for Lodash users

Given this array:

let family = [
  {
    name: "Mike",
    age: 10
  },
  {
    name: "Matt",
    age: 13
  },
  {
    name: "Nancy",
    age: 15
  },
  {
    name: "Adam",
    age: 22
  },
  {
    name: "Jenny",
    age: 85
  },
  {
    name: "Nancy",
    age: 2
  },
  {
    name: "Carl",
    age: 40
  }
]

1. Find duplicates:

let duplicatesArr = _.difference(family, _.uniqBy(family, 'name'), 'name')

// duplicatesArr:
// [{ 
//   name: "Nancy", 
//   age: 2 
// }]

2 Find if there are duplicates, for validation purpose:

let uniqArr = _.uniqBy(family, 'name')

if (uniqArr.length === family.length) {
    // No duplicates
}

if (uniqArr.length !== family.length) {
    // Has duplicates
}

Answer 6

Since most of the answers won't have a good performance, i thought i share my take on this:

const arrayWithDuplicateData = [{ id: 5, name: 'Facebook'}, { id: 3, name: 'Twitter' }, { id: 5, name: 'Facebook' }];


const uniqueObj = {};

arrayWithDuplicateData.forEach(i => {
  uniqueObj[i.id] = i;
});

const arrayWithoutDuplicates = Object.values(uniqueObj);

We're leveraging the fact that keys are unique within objects. That means the last duplication item inside the first array, will win over its predecessors. If we'd want to change that, we could flip the array before iterating over it.

Also we're not bound to use only one property of our object for identifying duplications.

const arrayWithDuplicateData = [{ id: 5, name: 'Facebook'}, { id: 3, name: 'Twitter' }, { id: 5, name: 'Facebook' }];


const uniqueObj = {};

arrayWithDuplicateData.forEach(item => {
  uniqueObj[`${item.id}_${item.name}`] = item;
});

const arrayWithoutDuplicates = Object.values(uniqueObj);

Or we could simply add a check, if the uniqueObj already holds a key and if yes, not overwrite it.

Overall this way is not very costly in terms of performance and served me well so far.

Answer 7

I would probably set up some kind of object. Since you've said ECMAScript 6, you have access to Set, but since you want to compare values on your objects, it will take a little more work than that.

An example might look something like this (removed namespace pattern for clarity):

var setOfValues = new Set();
var items = [];

function add(item, valueGetter) {
   var value = valueGetter(item);
   if (setOfValues.has(value))
      return;

   setOfValues.add(value);
   items.push(item);
}

function addMany(items, valueGetter) {
   items.forEach(item => add(item, valueGetter));
}

Use it like this:

var family = [
...
];

addMany(family, item => item.name);

// items will now contain the unique items

Explanation: you need to pull a value from each object as it's added and decide if it has already been added yet, based on the value you get. It requires a value getter, which is a function that given an item, returns a value (item => item.name). Then, you only add items whose values haven't already been seen.

---

A class implementation:

// Prevents duplicate objects from being added

class ObjectSet {
  constructor(key) {
    this.key = key;
    this.items = [];
    this.set = new Set();
  }

  add(item) {
    if (this.set.has(item[this.key])) return;
    this.set.add(item[this.key]);
    this.items.push(item);
  }

  addMany(items) {
    items.forEach(item => this.add(item));
  }
}

var mySet = new ObjectSet('name');
mySet.addMany(family);
console.log(mySet.items);