Pandas guess delimiter with sep=None

Question

Pandas documentation has this:

With sep=None, read_csv will try to infer the delimiter automatically in some cases by “sniffing”.

How can I access pandas' guess for the delimiter?

I want to read in 10 lines of my file, have pandas guess the delimiter, and start up my GUI with that delimiter already selected. But I don't know how to access what pandas thinks is the delimiter.

Also, is there a way to pass pandas a list of strings to restrict it's guesses to?

Answer 1

Looking at the source code, I doubt that it's possible to get the delimiter out of read_csv. But pandas internally uses the Sniffer class from the csv module. Here's an example that should get you going:

import csv
s = csv.Sniffer()
print s.sniff("a,b,c").delimiter
print s.sniff("a;b;c").delimiter
print s.sniff("a#b#c").delimiter

Output:

,
;
#

What remains, is reading the first line from a file and feeding it to the Sniffer.sniff() function, but I'll leave that up to you.

Answer 2

The csv.Sniffer is the simplest solution, but it doesn't work if you need to use compressed files. Here's what's working, although it uses a private member, so beware:

reader = pd.read_csv('path/to/file.tar.gz', sep=None, engine='python', iterator=True)
sep = reader._engine.data.dialect.delimiter
reader.close()