How to merge arrays of objects while removing duplicates

Question

I have a requirement to compare the two object arrays and remove duplicate and merge them into a single one. I am kind of new to javascript/Typescript. I have read some blogs which are using Map, reduce,filter to do the job. Is that the right approach?

However, I am wondering if I can create a map of it and the Map.ContainsKey or Map.getvalues();

In the real problem, ojects are quite huge, has several keys and even array has hundreds of records.

   [
    {
        Name: Roger
        Country : spain
    },
    {
        Name:Tiger
        Counry : USA
    },
    {
        Name: Micheal
        Country: USA
    },
]


[
    {
        Name: sachin
        Country : India
    },
    {
        Name:Roger
        Counry : Spain
    },
]

output 

[
    {
        Name: Roger
        Country : spain
    },
    {
        Name:Tiger
        Counry : USA
    },
    {
        Name: Micheal
        Country: USA
    },
    {
        Name: sachin
        Country : India
    }
]

Answer 1

You could take a hash table for the wanted values as key and push the elements which do not have a flag in the hash table.

var array1 = [{ Name: 'Roger', Country: 'Spain' }, { Name: 'Tiger', Counry: 'USA' }, { Name: 'Micheal', Country: 'USA' }],
    array2 = [{ Name: 'sachin', Country: 'India' }, { Name: 'Roger', Country: 'Spain' }],
    hash = Object.create(null),
    result = [array1, array2].reduce(function (r, a) {
        a.forEach(function (o) {
            var key = ['Name', 'Country'].map(k => o[k]).join('|');
            if (!hash[key]) {
                r.push(o);
                hash[key] = true;
            }
        });
        return r;
    }, []);

console.log(result);

.as-console-wrapper { max-height: 100% !important; top: 0; }

Answer 2

Es6 based solution using map() and filter() method.

Merge two array of objects and create one removing duplicate objects

let arr1 =    [
    {
        Name: 'Roger',
        Country : 'spain'
    },
    {
        Name:'Tiger',
        Country : 'USA'
    },
    {
        Name: 'Micheal',
        Country: 'USA'
    },
];
let arr2 = [
    {
        Name: 'sachin',
        Country : 'India'
    },
    {
        Name:'Roger',
        Country : 'Spain'
    },
]
let arr3 = [...arr1, ...arr2]

function mergeUniqueArray(arr, comp) {

  let unique = arr
    .map(item => item[comp])

     // store the keys of the unique objects
    .map((item, i, final) => final.indexOf(item) === i && i)

    // eliminate the duplicate keys & store unique objects
    .filter(item => arr[item]).map(item => arr[item]);
   return unique;
}

console.log(mergeUniqueArray(arr3,'Name'));

Answer 3

Here's one possible solution:

// Step 1, concat the two arrays together.
const allItems = items1.concat(items2);

// Step 2, map the above into an array of key/value entries.
const entries = allItems.map(o => [`${o.Name}:${o.Country}`, o]);

// Step 3, create a Map on the entries. De-duping happens here.
const map = new Map(entries);

// Step 4, extract the map's values back into an array.
const result = [...map.values()];

Some notes about this approach:

1. During the concat() in step 1, order matters! Items in items2 take precedence over items1. In other words, items at the end of the allItems array take precedence because later entries in the Map constructor overwrite previous ones. This matters if these objects have other properties besides the ones being compared.
2. It's remotely possible that certain name/country combos could collide if they contain the : character. You can get fancy with how you build the entry keys, if this becomes a problem.
3. Map uses a hashtable behind the scenes, so the de-duplication should be reasonably fast.
4. By design, Map remembers input order, so the final array will reflect the same order as allItems, but with duplicates removed. For a different order, you'll need to sort.