Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

unicode datas of a dataframe to strings

I have some troubles with a dataframe obtained from reading a xls file. Every data on such dataframe has the type 'unicode' and I can't do anything with this. I wanna change it to str values. Also, iff possible, I'd like to know the reason of this fact. I heard something about 'external data', and I know that both columns and index also present the 'u' of unicode before the names of these ones. I don't know neither almost anything about encoding and I would be really grateful if someone explains something about this in addition.

I'm using Python 2 and I tryed to solve it column by column with functions as

.astype(str) 
.astype(basestring)
.apply(str) 

and

.str.decode('iso-8859-1').str.encode('utf-8') 

(I read this last one here and I just wrote it in my code to try another thing). I also tried

unicodedata.normalize('NFKD', df_bolsa[l]).encode('ascii','ignore')

but this last one cannot be used with a series. I hope someone to be able to help me to clarify this matter. Thank you very much in advance!!

like image 401
emilio.molina Avatar asked Feb 23 '17 17:02

emilio.molina


People also ask

How do you convert a DataFrame to a string?

If you want to change the data type for all columns in the DataFrame to the string type, you can use df. applymap(str) or df. astype(str) methods.

How can I get single value as a string from pandas data frame?

Try using t = df[df['Host'] == 'a']['Port'][0] or t = df[df['Host'] == 'a']['Port'][1] .


2 Answers

You can use the following code.

for column in df:
    df[column] = df_peru[column].str.encode('utf-8')
like image 187
emilio.molina Avatar answered Sep 17 '22 23:09

emilio.molina


To help others, this version worked for me. I was getting an error while loading my dataframe to an oracle database: "UnicodeDecodeError: 'ascii' codec can't decode byte 0xea in position 2: ordinal not in range(128)"

I am on Python ver 2.7

for column in df:
    df[column]=  df[column].astype(str).str.decode('utf-8')
like image 34
MEdwin Avatar answered Sep 19 '22 23:09

MEdwin