Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas to_csv(sys.stdout) doesn't work under my environment

I ran a following simple script:

import sys
import pandas as pd
df = pd.DataFrame([[1,2,3], [4,5,6]])
df.to_csv(sys.stdout)

Expected Output (as standard output)

,0,1,2
0,1,2,3
1,4,5,6

However, I got no outputs on standard output by the program under Python 2.7.15 on macOS 10.12.6. Instead, it generated a file named <stdout> which contains the expected output.

Interestingly, on the same OS, Python 3.6.5 could show the result as standard output without any problems, and virtualenved Python 3.6.5 could not show it (and generated the <stdout> file).

Does anyone identify the cause of this result? The version of Pandas is 0.23.1.

like image 891
kiichi Avatar asked Jul 06 '18 00:07

kiichi


People also ask

Does pandas to_csv overwrite?

Does pandas To_csv overwrite? If the file already exists, it will be overwritten. If no path is given, then the Frame will be serialized into a string, and that string will be returned.

Does to_csv create directory?

to_csv does create the file if it doesn't exist as you said, but it does not create directories that don't exist. Ensure that the subdirectory you are trying to save your file within has been created first. This can easily be wrapped up in a function if you need to do this frequently.

How do I write pandas DataFrame to CSV?

To write a pandas DataFrame to a CSV file, you will need DataFrame. to_csv . This function offers many arguments with reasonable defaults that you will more often than not need to override to suit your specific use case.


1 Answers

Depending on what you are ultimately trying to do this workaround might or might not be useful:

In [85]: from io import StringIO

In [86]: output = StringIO()

In [87]: df.to_csv(output)

In [88]: print(output.getvalue())
,0,1,2
0,1,2,3
1,4,5,6

(Python 3.6)

For Python 2.7 replace the one line above with:

from StringIO import StringIO

UPDATE:

Actually, I think this might be the 'proper' way to do it:

In [3]: from io import StringIO

In [4]: output = StringIO()

In [5]: df.to_csv(output)

In [6]: output.seek(0)
Out[6]: 0

In [7]: print(output.read())
,0,1,2
0,1,2,3
1,4,5,6
like image 81
Bill Avatar answered Sep 17 '22 18:09

Bill