Find the common string in a subgroup in a list in Python

Question

I'm trying to clean a list, by removing duplicates. For example:

 bb = ['Gppe (Aspirin Combined)', 
       'Gppe Cap (Migraine)',  
       'Gppe Tab', 
       'Abilify', 
       'Abilify Maintena', 
       'Abstem', 
       'Abstral']

Ideally, I need to get the following list:

 bb = ['Gppe', 
       'Abilify', 
       'Abstem', 
       'Abstral']

What I tried:

1. Split the list and remove duplicates (a naive approach)

   list(set(sorted([j for bb_i in bb for j in bb_i.split(' ')])))

which leaves a lot of 'rubbish':

['(Aspirin',
 '(Migraine)',
 'Abilify',
 'Abstem',
 'Abstral',
 'Cap',
 'Combined)',
 'Gppe',
 'Maintena',
 'Tab']

2. Find the most frequent word:

Counter(['Gppe (Aspirin Combined)', 'Gppe Cap (Migraine)', 'Gppe Tab').most_common(1)[0][0]

But I'm not sure how to find similar words (a group)??

I am wondering, whether one can use a kind of 'groupby()' and first group by names and then remove duplicates within those names.

Answer 1

You could do, assuming you want the unique first word of each string:

bb = ['Gppe (Aspirin Combined)',
       'Gppe Cap (Migraine)',
       'Gppe Tab',
       'Abilify',
       'Abilify Maintena',
       'Abstem',
       'Abstral']


result = set(map(lambda x: x.split()[0], bb))
print(result)

Output

{'Gppe', 'Abstral', 'Abilify', 'Abstem'}

If you want a list of unique elements in the order of appearance, you could do:

bb = ['Gppe (Aspirin Combined)',
       'Gppe Cap (Migraine)',
       'Gppe Tab',
       'Abilify',
       'Abilify Maintena',
       'Abstem',
       'Abstral']

seen = set()
result = []
for e in bb:
    key = e.split()[0]
    if key not in seen:
        result.append(key)
        seen.add(key)

print(result)

Output

['Gppe', 'Abilify', 'Abstem', 'Abstral']

As an alternative to the first solution you could do:

1. Suggested by @Jean-FrançoisFabre {x.split()[0] for x in bb}
2. Suggested by @RoadRunner set(x.split()[0] for x in bb)