Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a possibility to keep column order when reading parquet?

Saving a dataframe with columns (e.g. "a", "b") as parquet and then reading the parquet at later point in time does not deliver the same column order (could be "b", "a" f.e.) as the file was saved with.

Unfortunately, I was not able to figure out, how the order is influenced and how I can control it.

How to keep original column order when reading in parquet?

like image 208
Martin Senne Avatar asked Sep 18 '25 20:09

Martin Senne


1 Answers

PARQUET-188 suggests that column ordering is not part of the parquet spec, so it's probably not a good idea to rely on the ordering. You could however manage this yourself, e.g. by loading/saving the dataframe columns in lexicographical order, or by storing the column names.

like image 192
dpeacock Avatar answered Sep 21 '25 11:09

dpeacock