Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas "read_csv" Function Returns NAN for All Blocks in My Table

Tags:

python

pandas

csv

My original CSV file has float64 values in each block but after I use pd.csv() to read the file, it returns me a blank table. I tried to set the delimiter and the encoding of the function but it didn't help at all. The CSV file is automatically generated by a software and I have no way check the settings of the settings. Is there any way I can read my file to a dataframe with correct values?

>>> pd.read_csv('./HISTORY_LOG_05-31-2018.CSV')
D  Unnamed: 1  Unnamed: 2      ...       Unnamed: 108  Unnamed: 109  Unnamed: 110
0 NaN         NaN         NaN      ...                NaN           NaN           NaN
1 NaN         NaN         NaN      ...                NaN           NaN           NaN
2 NaN         NaN         NaN      ...                NaN           NaN           NaN
3 NaN         NaN         NaN      ...                NaN           NaN           NaN
4 NaN         NaN         NaN      ...                NaN           NaN           NaN
5 NaN         NaN         NaN      ...                NaN           NaN           NaN

[6 rows x 111 columns]

I simplified the CSV file to

A,B
0.000,0.000

0.000,0.000

and I still got results like:

>>> pd.read_table('./HISTORY_LOG_05-31-2018.CSV', encoding="cp1252")
    D
0 NaN
1 NaN
2 NaN

>>> pd.read_table('./HISTORY_LOG_05-31-2018.CSV', encoding="cp1252", delimiter=",")
    D  Unnamed: 1
0 NaN         NaN
1 NaN         NaN
2 NaN         NaN
like image 894
オウエキセツ Avatar asked Jun 21 '18 18:06

オウエキセツ


1 Answers

So, I figured out the answer as I had this same problem. My encoding was wrong and so it wouldn't read the text correctly. I opened it in Visual Studio Code and found the encoding was UTF-16 LE. My output came from powershell so yours likely did too and you probably just need to specify the output encoding or change the encoding for panda.

pd.read_csv("ADSearch.txt",encoding='UTF-16 LE')
Empty DataFrame
Columns: [lastname, firstname, username, site, email, Unnamed: 5, False, True]
Index: []
like image 200
Christopher Avatar answered Oct 30 '22 21:10

Christopher