Console warnings to file

Question

I am running a python script using pandas.read_csv to import a csv file. Pandas provides console warnings when it doesn't see what it expects such as:

Skipping line 163: Expected 41 fields in line 163, saw 42

How can I log this to a text file?

If I run the script from a command line, python > logfile.txt only the output of print shows up in the file, not the warnings.

Answer 1

Use stderr

In bash, there are 3 default streams:

STDOUT: Redirected by > or 1>, this is standard output from a program.

STDERR: Redirected by 2>, this is diagnostic output from a program.

STDIN: Input from console, use < to input it.

./prog 2> errorlog.txt >logfile.txt

If you want to redirect ALL output to STDOUT, use:

./prog 2>&1

Here is some more information: I/O Redirection