Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to remove commas from ALL the column in pandas at once

I have a data frame where all the columns are supposed to be numbers. While reading it, some of them were read with commas. I know a single column can be fixed by

df['x']=df['x'].str.replace(',','')

However, this works only for series objects and not for entire data frame. Is there an elegant way to apply it to entire data frame since every single entry in the data frame should be a number.

P.S: To ensure I can str.replace, I have first converted the data frame to str by using

df.astype('str')

So I understand, I will have to convert them all to numeric once the comma is removed.

like image 484
PagMax Avatar asked Jul 09 '19 07:07

PagMax


People also ask

How do pandas remove commas?

Use the str. replace() method to remove the punctuation from the string. Use the str. split() method to split on whitespace characters.

How do you remove all commas in Python?

sub() function to erase commas from the python string. The function re. sub() is used to swap the substring. Also, it will replace any match with the other parameter, in this case, the null string, eliminating all commas from the string.

How do I strip all column values in pandas?

Pandas provide predefine method “pandas. Series. str. strip()” to remove the whitespace from the string.

How do I remove a comma from a string column in Python?

Use str. replace() to remove a comma from a string in Python replace(',', '') to replace every instance of a ',' in str with '' .


2 Answers

Numeric columns have no ,, so converting to strings is not necessary, only use DataFrame.replace with regex=True for substrings replacement:

df = df.replace(',','', regex=True)

Or:

df.replace(',','', regex=True, inplace=True)

And last convert strings columns to numeric, thank you @anki_91:

c = df.select_dtypes(object).columns
df[c] = df[c].apply(pd.to_numeric,errors='coerce')
like image 121
jezrael Avatar answered Oct 17 '22 14:10

jezrael


Well, you can simplely do:

df = df.apply(lambda x: x.str.replace(',', ''))

Hope it helps!

like image 32
Azuuu Avatar answered Oct 17 '22 14:10

Azuuu