Pandas usecols all except last

Question

I have a csv file, is it possible to have usecols take all columns except the last one when utilizing read_csv without listing every column needed.

For example, if I have a 13 column file, I can do usecols=[0,1,...,10,11]. Doing usecols=[:-1] will give me syntax error?

Is there another alternative? I'm using pandas 0.17

Answer 1

Starting from version 0.20 the usecols method in pandas accepts a callable filter, i.e. a lambda expression. Hence if you know the name of the column you want to skip you can do as follows:

columns_to_skip = ['foo','bar']
df = pd.read_csv(file, usecols=lambda x: x not in columns_to_skip )

Here's the documentation reference.

Answer 2

You can just read a single line using nrows=1 to get the cols and then re-read in the full csv skipping the last col by slicing the column array from the first read:

cols = pd.read_csv(file, nrows=1).columns
df = pd.read_csv(file, usecols=cols[:-1])