Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it better to store Pandas Data Frames in a dictionary or in a Panel?

I hope this doesn't sound as an open question for discussion. I am going to give some details for my specific case.

I am new to Pandas and I need to store several 2D arrays, where columns represent frequencies and rows represent directions (2D waves spectra, if you are curious). Each array represent a specific time.

I am storing these arrays as Pandas DataFrames, but for keeping them in a single object I thought of 2 options:

  1. Storing the DataFrames in a dictionary where the key is the time stamp.

  2. Storing the DataFrames in a Pandas Panel where the item is the time stamp.

The first option seems simple and has the flexibility to store arrays with different sizes, indexes and column names. The second option seems better for processing the data, since Panels have specific methods, and can also be easily saved or exported (e.g. to csv or pickle).

Which of the two options is better suited in terms of: speed, memory use, flexibility and data analysis?

Regards

like image 977
jcdoming Avatar asked Nov 09 '22 19:11

jcdoming


1 Answers

I don't think you need a panel. I recommend a nested dataframe approach.

like image 109
kilojoules Avatar answered Nov 14 '22 21:11

kilojoules