Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Remove rows that two columns have the same values by pandas

Tags:

pandas

Input:

    S   T   W      U
0   A   A   1   Undirected
1   A   B   0   Undirected
2   A   C   1   Undirected
3   B   A   0   Undirected
4   B   B   1   Undirected
5   B   C   1   Undirected
6   C   A   1   Undirected
7   C   B   1   Undirected
8   C   C   1   Undirected

Output:

    S   T   W      U
1   A   B   0   Undirected
2   A   C   1   Undirected
3   B   A   0   Undirected
5   B   C   1   Undirected
6   C   A   1   Undirected
7   C   B   1   Undirected

For column S and T ,rows(0,4,8) have same values. I want to drop these rows.

Trying:

I used df.drop_duplicates(['S','T'] but failed, how could I get the results.

like image 604
Jack Avatar asked May 13 '17 09:05

Jack


People also ask

How do I delete duplicate rows based on multiple columns in pandas?

By using pandas. DataFrame. drop_duplicates() method you can drop/remove/delete duplicate rows from DataFrame. Using this method you can drop duplicate rows on selected multiple columns or all columns.

How do you delete a row from a DataFrame based on multiple column values?

Use drop() method to delete rows based on column value in pandas DataFrame, as part of the data cleansing, you would be required to drop rows from the DataFrame when a column value matches with a static value or on another column value.

How do you drop duplicate rows in pandas based on a column?

To remove duplicates of only one or a subset of columns, specify subset as the individual column or list of columns that should be unique. To do this conditional on a different column's value, you can sort_values(colname) and specify keep equals either first or last .


1 Answers

You need boolean indexing:

print (df['S'] != df['T'])
0    False
1     True
2     True
3     True
4    False
5     True
6     True
7     True
8    False
dtype: bool

df = df[df['S'] != df['T']]
print (df)
   S  T  W           U
1  A  B  0  Undirected
2  A  C  1  Undirected
3  B  A  0  Undirected
5  B  C  1  Undirected
6  C  A  1  Undirected
7  C  B  1  Undirected

Or query:

df = df.query("S != T")
print (df)
   S  T  W           U
1  A  B  0  Undirected
2  A  C  1  Undirected
3  B  A  0  Undirected
5  B  C  1  Undirected
6  C  A  1  Undirected
7  C  B  1  Undirected
like image 139
jezrael Avatar answered Oct 06 '22 08:10

jezrael