the order within group apply function

Question

Having the code (below) I am trying to figure will particular group order always remain the same as in original dataframe.

It looks like the order within the group preserved for my little example, but what if I have dataframe with ~1 mln records? Will pandas provide such guarantee and I should worry about that by myself?

Code:

import numpy as np
import pandas as pd

N = 10
df = pd.DataFrame(index = xrange(N))
df['A'] = map(lambda x: int(x) / 5, np.random.randn(N) * 10.0)
df['B'] = map(lambda x: int(x) / 5, np.random.randn(N) * 10.0)
df['v'] = np.random.randn(N)

def show_x(x):
    print x
    print "----------------"
df.groupby('A').apply(show_x)
print "==============="
print df

Output:

   A  B         v
6 -4 -1 -2.047354

[1 rows x 3 columns]
----------------
   A  B         v
6 -4 -1 -2.047354

[1 rows x 3 columns]
----------------
   A  B         v
8 -3  0 -1.190831

[1 rows x 3 columns]
----------------
   A  B         v
0 -1 -1  0.456397
9 -1 -2 -1.329169

[2 rows x 3 columns]
----------------
   A  B         v
1  0  0  0.663928
2  0  2  0.626204
7  0 -3 -0.539166

[3 rows x 3 columns]
----------------
   A  B         v
4  2  2 -1.115721
5  2  1 -1.905266

[2 rows x 3 columns]
----------------
   A  B         v
3  4 -1  0.751016

[1 rows x 3 columns]
----------------
===============
   A  B         v
0 -1 -1  0.456397
1  0  0  0.663928
2  0  2  0.626204
3  4 -1  0.751016
4  2  2 -1.115721
5  2  1 -1.905266
6 -4 -1 -2.047354
7  0 -3 -0.539166
8 -3  0 -1.190831
9 -1 -2 -1.329169

[10 rows x 3 columns]

Answer 1

If you are using apply not only is the order not guaranteed, but as you've found it can trigger the function for the same group a couple of times (to decide which "path" to take / what type of result to return). So if your function has side-effects don't do this!

I recommend simply iterating through the groupby object!

In [11]: df = pd.DataFrame([[1, 2], [1, 4], [5, 6]], columns=['A', 'B'])

In [12]: df
Out[12]:
   A  B
0  1  2
1  1  4
2  5  6

In [13]: g = df.groupby('A')

In [14]: for key, sub_df in g:
             print("key =", key)
             print(sub_df)
             print('')   # apply whatever function you want
key = 1
   A  B
0  1  2
1  1  4

key = 5
   A  B
2  5  6

Note that this is ordered (the same as the levels) see g.grouper._get_group_keys():

In [21]: g.grouper.levels
Out[21]: [Int64Index([1, 5], dtype='int64')]

It's sorted by default (there's a sort kwarg when doing the groupby), through it's not clear what this actually means if it's not a numeric dtype.