Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Use TQDM Progress Bar with Pandas

Is it possible to use TQDM progress bar when importing and indexing large datasets using Pandas?

Here is an example of of some 5-minute data I am importing, indexing, and using to_datetime. It takes a while and it would be nice to see a progress bar.

#Import csv files into a Pandas dataframes and convert to Pandas datetime and set to index  eurusd_ask = pd.read_csv('EURUSD_Candlestick_5_m_ASK_01.01.2012-05.08.2017.csv') eurusd_ask.index = pd.to_datetime(eurusd_ask.pop('Gmt time')) 
like image 879
sslack88 Avatar asked Nov 03 '17 02:11

sslack88


People also ask

What is tqdm progress bar?

tqdm derives from the Arabic word taqaddum (تقدّم) which can mean "progress," and is an abbreviation for "I love you so much" in Spanish (te quiero demasiado). Instantly make your loops show a smart progress meter - just wrap any iterable with tqdm(iterable) , and you're done!


1 Answers

Find length by getting shape

for index, row in tqdm(df.iterrows(), total=df.shape[0]):    print("index",index)    print("row",row) 
like image 165
Arjun Kava Avatar answered Sep 20 '22 12:09

Arjun Kava