Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Drop duplicates for rows with interchangeable name values (Pandas, Python)

I have a DataFrame of form

person1, person2, ..., someMetric
John, Steve, ..., 20
Peter, Larry, ..., 12
Steve, John, ..., 20

Rows 0 and 2 are interchangeable duplicates, so I'd want to drop the last row. I can't figure out how to do this in Pandas.

Thanks!

like image 554
Daniel Messias Avatar asked Sep 08 '26 06:09

Daniel Messias


2 Answers

Here's a NumPy based solution -

df[~(np.triu(df.person1.values[:,None] == df.person2.values)).any(0)]

Sample run -

In [123]: df
Out[123]: 
  person1 person2 someMetric
0    John   Steve         20
1   Peter   Larry         13
2   Steve    John         19
3   Peter  Parker          5
4   Larry   Peter          7

In [124]: df[~(np.triu(df.person1.values[:,None] == df.person2.values)).any(0)]
Out[124]: 
  person1 person2 someMetric
0    John   Steve         20
1   Peter   Larry         13
3   Peter  Parker          5
like image 137
Divakar Avatar answered Sep 10 '26 18:09

Divakar


an approach in pandas

df = pd.DataFrame(
{'person2':  {0: 'Steve', 1: 'Larry', 2: 'John', 3: 'Parker', 4: 'Peter'}, 
'person1': {0: 'John', 1: 'Peter', 2: 'Steve', 3: 'Peter', 4: 'Larry'}, 
'someMetric': {0: 20, 1: 13, 2: 19, 3: 5, 4: 7}})


print(df)
  person1 person2 someMetric
0    John   Steve         20
1   Peter   Larry         13
2   Steve    John         19
3   Peter  Parker          5
4   Larry   Peter          7


df['ordered-name'] = df.apply(lambda x: '-'.join(sorted([x['person1'],x['person2']])),axis=1)
df = df.drop_duplicates(['ordered-name'])
df.drop(['ordered-name'], axis=1, inplace=True)
print df

which gives:

  person1 person2  someMetric
0    John   Steve          20
1   Peter   Larry          13
3   Peter  Parker           5
like image 31
Dickster Avatar answered Sep 10 '26 19:09

Dickster



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!