Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Split pandas dataframe in two if it has more than 10 rows

I have a huge CSV with many tables with many rows. I would like to simply split each dataframe into 2 if it contains more than 10 rows.

If true, I would like the first dataframe to contain the first 10 and the rest in the second dataframe.

Is there a convenient function for this? I've looked around but found nothing useful...

i.e. split_dataframe(df, 2(if > 10))?

like image 334
Boosted_d16 Avatar asked Aug 13 '14 15:08

Boosted_d16


People also ask

How do I separate rows in pandas?

To split cell into multiple rows in a Python Pandas dataframe, we can use the apply method. to call apply with a lambda function that calls str. split to split the x string value. And then we call explode to fill new rows with the split values.


1 Answers

I used a List Comprehension to cut a huge DataFrame into blocks of 100'000:

size = 100000 list_of_dfs = [df.loc[i:i+size-1,:] for i in range(0, len(df),size)] 

or as generator:

list_of_dfs = (df.loc[i:i+size-1,:] for i in range(0, len(df),size)) 
like image 181
agittarius Avatar answered Sep 29 '22 13:09

agittarius