Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

I always get a Kernel Dead when using "pd.read_parquet()". (No matter which file size)

As you can guess by the title, the kernel always dies when using pd.read_parquet(). I already tried it with different sizes but it wont work.

here the code... (I am using jupyter (without anaconda, because it always takes to long to start) in Python 3.7 with a i5 & 16GB RAM)

outfp = PurePath(data_dir+'/interim/IVE_tickbidask.parq') 
#df = df.head(10)
df.to_parquet(outfp)

from pathlib import PurePath, Path
import pandas as pd
data_dir = "../../../Adv_Fin_ML_Exercises-master/Adv_Fin_ML_Exercises-master/data"
infp=PurePath(data_dir+'/interim/IVE_tickbidask.parq')
df = pd.read_parquet(data_dir+'/interim/IVE_tickbidask.parq')
cprint(df)

What can i do to still make it work?

like image 818
Benoid Avatar asked Oct 24 '25 08:10

Benoid


1 Answers

I had the same problem, adding engine = 'fastparquet' worked for me. Otherwise it defaults to engine = 'pyarrow' and that seems to make the kernel die.

like image 198
NatalieL Avatar answered Oct 27 '25 01:10

NatalieL