Compare dicts and merge them. No overwrite and no duplicate values

Question

I made a mistake in my question here (wrong requested input and expected output): Comparing dicts, updating NOT overwriting values

I am not looking for this solution: Combining 2 dictionaries with common key So this question is not a duplicate

Problem statement:

requested input:

d1 = {'a': ['a'], 'b': ['b', 'c']}
d2 = {'b': ['c', 'd'], 'c': ['e','f']}

expected output (I don't care about the order of the keys / values!):

new_dict = {'a': ['a'], 'b': ['b', 'c', 'd'], 'c': ['e', 'f']}

The solution in Combining 2 dictionaries with common key gives following output:

new_dict = {'a': ['a'], 'b': ['b', 'c', 'c', 'd'], 'c': ['e', 'f']}

I don't want the duplicates to be stored.

My solution (it works but it is not so efficient):

unique_vals = []
new_dict = {}
for key in list(d1.keys())+list(d2.keys()) :
    unique_vals = []
    try:
        for val in d1[key]:
            try:
                for val1 in d2[key]:
                    if(val1 == val) and (val1 not in unique_vals):
                        unique_vals.append(val)
            except:
                continue
    except:        
        new_dict[key] = unique_vals
    new_dict[key] = unique_vals


for key in d1.keys():
    for val in d1[key]:
        if val not in new_dict[key]:
            new_dict[key].append(val)
for key in d2.keys():
    for val in d2[key]:
        if val not in new_dict[key]:
            new_dict[key].append(val)

Answer 1

Here is how I would go about it:

d1 = {'a': ['a'], 'b': ['b', 'c']}
d2 = {'b': ['c', 'd'], 'c': ['e','f']}
dd1 = {**d1, **d2}
dd2 = {**d2, **d1}
{k:list(set(dd1[k]).union(set(dd2[k]))) for k in dd1}

Produces the desired result.

Answer 2

I suggest using a default dictionary collection with a set as a default value. It guarantees that all values will be unique and makes the code cleaner.

Talking about efficiecy it's O(n^2) by time.

   from collections import defaultdict


   d1 = {'a': ['a'], 'b': ['b', 'c']}
   d2 = {'b': ['c', 'd'], 'c': ['e','f']}

   new_dict = defaultdict(set)

   for k, v in d1.items():
       new_dict[k] = new_dict[k].union(set(v))

   for k, v in d2.items():
       new_dict[k] = new_dict[k].union(set(v))