8 Ways to Remove Duplicate Array Values in JavaScript

1. De-Dupe Arrays with Set Objects

Set objects objects are my go-to when de-duplicating arrays. Try out this example in your browser console,

function removeDuplicateArrayValues(arr) {
  return [...new Set(arr)];
}

const alpineFauna = ["marmot", "bear", "marmot", "bear", "lynx"];

console.log(removeDuplicateArrayValues(alpineFauna));
// Array(3) [ "marmot", "bear", "lynx" ]

const alpineFlora = ["buttercup", "agrimony", "agrimony", "buttercup"];

console.log(removeDuplicateArrayValues(alpineFlora));
// Array(2) [ "buttercup", "agrimony" ]

• spoiler - Set objects work great for primitive types, but may not support use cases requiring the de-duplication of objects in an array.

Set De-Dupe in Depth

Similar to JavaScript arrays, Set objects are collections of values. But, that's pretty much where the similarities end. The Array and Set objects' interfaces are completely different from each other - Set objects lack the standard Array class methods.

JavaScript Set objects are useful in that, unlike arrays, values within a Set must be unique - values can only occur one time. This attribute makes Set objects great for removing duplicate values from arrays.

Check out this benign example,

const alpineFauna = ["marmot", "bear", "marmot", "bear", "lynx"];

console.log(alpineFauna); // Array(5) [ "marmot", "bear", "marmot", "bear", "lynx" ]

const mySet = new Set(["marmot", "bear", "marmot", "bear", "lynx"]);

console.log(mySet); // Set [ "marmot", "bear", "lynx" ]

While we can instantiate a new Set object instance when creating a data structure from scratch, other times call for the de-duplication of an existing array.

How to convert an array to a Set object

Like the above examples, simply pass an array to the Set object constructor when instantiating a new instance,

const mySet = new Set(__MY_ARRAY__);

Converting a Set object to an array is slightly trickier, but it isn't a bear 😜

How to convert a ES6 Set object to an array

There are two ways to convert a Set object to an array. The first makes use of Array.from(), a native static method that allows us to create shallow-copied arrays from array-like objects.

const mySet = new Set(["marmot", "bear", "marmot", "bear", "lynx"]);

const myArray = Array.from(mySet);

console.log(myArray); // Array(3) [ "marmot", "bear", "lynx" ]

Alternatively, we can use spread syntax to concisely convert a Set object instance to an array,

const mySet = new Set(["marmot", "bear", "marmot", "bear", "lynx"]);

const myArray = [...mySet];

console.log(myArray); // Array(3) [ "marmot", "bear", "lynx" ]

What types does Set de-duplicate?

Removing duplicate values from an array using Set works well for primitive value types and object instances - Set removes duplicate numbers & strings and array & object instances. Set won't work when trying to remove duplicates in situations like the following,

const mySetOfObjects = new Set([
  { type: "dog" },
  { type: "dog" },
  { type: "dog" },
]);

console.log(mySetOfObjects);

// Set(3) [ { "type": "dog" }, { "type": "dog" }, { "type": "dog" } ]

const mySetOfArrays = new Set([
  [1, 2, 3],
  [1, 2, 3],
  [1, 2, 3],
]);

console.log(mySetOfArrays);
// Set(3) [ [ 1, 2, 3 ], [ 1, 2, 3 ], [ 1, 2, 3 ] ]

Use cases requiring an array of unique objects or arrays is best achieved using some combination of .find(), .filter(), and .reduce()

2. Remove Duplicate Objects from an Array Using .reduce()

Using the Setobject is fantastic for straightforward use cases. It's readable and low-code. Unfortunately, it can't be applied to more complex situations, such as when we want to ensure all of the objects in an existing array are unique.

Let's turn to .reduce() to remove duplicate products from an array below,

function removeDuplicateProducts(products) {
  return products.reduce((acc, product) => {
    const hasProduct = !!acc.find(
      (uniqueProduct) => uniqueProduct.id === product.id
    );

    if (!hasProduct) {
      return [...acc, product];
    }

    return acc;
  }, []);
}

const products = [
  { id: 1, type: "shirt", color: "blue" },
  { id: 1, type: "shirt", color: "blue" },
  { id: 2, type: "shirt", color: "yellow" },
  { id: 3, type: "shirt", color: "red" },
];

console.log(removeDuplicateProducts(products));
/*
  [
    { "id": 1, "type": "shirt"  "color": "blue" }
    { "id": 2, "type": "shirt", "color": "yellow" },
    { "id": 3, "type": "shirt", "color": "red" }
  ]
*/

Using .reduce to de-dupe an array of objects is among the more concise approaches. It's also among the more complicated.

The most concise approach isn't necessarily the best approach - consider the readability and maintainability when introducing code snippets from the internet into your project.

It's also important to consider this may not be the most efficient means of array de-duplication, as the size of the acc values increases with each unique product. It's unlikely such considerations are necessary for most real-world browser based use cases. But, it's worth keeping efficiency in mind when de-duping your arrays.

3. Remove Duplicates Objects from an Array Using .forEach()

De-duplication using .forEach() is similar to the above .reduce() example.

function removeDuplicateProducts(products) {
  const uniqueProducts = [];

  products.forEach((product) => {
    const hasProduct = !!uniqueProducts.find(
      (uniqueProduct) => uniqueProduct.id === product.id
    );

    if (!hasProduct) {
      uniqueProducts.push(product);
    }
  });

  return uniqueProducts;
}

const products = [
  { id: 1, type: "shirt", color: "blue" },
  { id: 1, type: "shirt", color: "blue" },
  { id: 2, type: "shirt", color: "yellow" },
  { id: 3, type: "shirt", color: "red" },
];

console.log(removeDuplicateProducts(products));
/*
  [
    { "id": 1, "type": "shirt"  "color": "blue" }
    { "id": 2, "type": "shirt", "color": "yellow" },
    { "id": 3, "type": "shirt", "color": "red" }
  ]
*/

These approaches work best when filtering values out of the final array based on a single (or a few) object attributes. Using .reduce() or .forEach() for deep object comparisons is tedious and may be better suited for a third party library, such as Lodash.

4. Remove Duplicate Values from an Array Using .filter()

Let's take a break from removing duplicate objects from an array and return to a simpler problem - removing duplicate values from an array. If the Set object de-duplication approach doesn't work for your use case, the .filter() method may,

function removeDuplicateArrayValues(arr) {
  return arr.filter((value, index) => arr.indexOf(value) === index);
}

const letters = ["a", "a", "a", "b", "b", "c", "c", "d", "x", "z", "z"];

console.log(removeDuplicateArrayValues(letters));
// Array(6) [ "a", "b", "c", "d", "x", "z" ]

Using .filter() and .indexOf() removes the duplicate array values in a succinct and readable manner, allowing us to get all of the unique values in the array without much overhead.

Finding Duplicated Values in an Array

In some cases, we may want to find all of the values in an array that occur more than once. This is easy with .filter() - all we need to do is change our condition from arr.indexOf(value) === index to arr.indexOf(value) !== index.

function removeDuplicateArrayValues(arr) {
  return arr.filter((value, index) => arr.indexOf(value) === index);
}

function getDuplicatedValues(arr) {
  const duplicatedValues = arr.filter(
    (value, index) => arr.indexOf(value) !== index
  );

  console.log(duplicatedValues); // Array(5) [ "a", "a", "b", "c", "z" ]

  /*
    Looking at the console.log of duplicatedValues above, we find there are
    multiple "a" characters in the array. Now that we've found all of the
    duplicate characters, we use function composition to return a
    de-duplicated array of duplicate characters 🤯
  */
  return removeDuplicateArrayValues(duplicatedValues);
}

const letters = ["a", "a", "a", "b", "b", "c", "c", "d", "x", "z", "z"];

console.log(getDuplicatedValues(letters)); // Array(4) [ "a", "b", "c", "z" ]

5. Extend the Array Prototype

Popular wisdom recommends developers avoid extending native object prototypes. But, maybe you eschew popular wisdom or have a great use case. If you really want to, here' show you can extend the native array prototype to get the unique values from an array,

Array.prototype.removeDuplicateArrayValues = function () {
  var uniqueValues = [];
  for (i = 0; i < this.length; i++) {
    if (uniqueValues.indexOf(this[i]) === -1) {
      uniqueValues.push(this[i]);
    }
  }

  return uniqueValues;
};

const letters = ["a", "a", "a", "b", "b", "c", "c", "d", "x", "z", "z"];

console.log(letters.removeDuplicateArrayValues());
// // Array(6) [ "a", "b", "c", "d", "x", "z" ]

The above example uses an "old school" JavaScript approach. Check out how much simpler this looks using the ES6 Set object,

Array.prototype.removeDuplicateArrayValues = function () {
  return [...new Set(this)];
};

const letters = ["a", "a", "a", "b", "b", "c", "c", "d", "x", "z", "z"];

console.log(letters.removeDuplicateArrayValues());
// // Array(6) [ "a", "b", "c", "d", "x", "z" ]

6. Keep it Classy 💎

As previously mentioned - popular wisdom recommends developers avoid extending native object prototypes.

There are a myriad of reasons, which I'll explore in a later article. For now, let's instead explore how we can create similar de-duplication functional using a ES6 class.

class ArrayMagic {
  constructor(array) {
    this.array = array;

    return this;
  }

  removeDuplicateValues() {
    return [...new Set(this.array)];
  }
}

const letters = ["a", "a", "a", "b", "b", "c", "c", "d", "x", "z", "z"];

const uniqueLetters = new ArrayMagic(letters).removeDuplicateValues();

console.log(uniqueLetters); // Array(6) [ "a", "b", "c", "d", "x", "z" ]

The ArrayMagic class only filters out primitive values (like strings and numbers). This class is provided as an example, and should be extending as necessary when simpler alternatives are unavailable.

Keep in mind there are other ways to create developer friendly Class interfaces, but we won't dig into them here.

7. Remove Duplicate Values with Lodash

Lodash's _.uniq(array) function creates a duplicate free version of the provided array argument.

If you're already using Lodash in your project, using the _.uniq() function may be a reasonable way to de-duplicate an array. If you're not, consider leveraging a native approach.

Either way, try out the lodash approach to array de-duplication at repl.it

const _ = require("lodash");

const letters = ["a", "a", "a", "b", "b", "c", "c", "d", "x", "z", "z"];

console.log(_.uniq(letters)); // [ 'a', 'b', 'c', 'd', 'x', 'z' ]

Unfortunately, this method doesn't filter out duplicated object structures from an array. It isn't a magic bullet. You'll need to author custom code to achieve such.

8. De-Dupe with Underscore.js

Underscore.js's _.uniq(array) function behaves similarly to Lodash's. It fact, it behaves so similarly you can refer to the Lodash example for a basic use case (optional parameters vary).

Projects unable to leverage ES6, and already leveraging Underscore, may wish to make use of this method. Otherwise, many will find the above approaches introduce less project overhead.