Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why doesn't first and last in a groupby give me first and last

I'm posting this because the topic just got brought up in another question/answer and the behavior isn't very well documented.

Consider the dataframe df

df = pd.DataFrame(dict(
    A=list('xxxyyy'),
    B=[np.nan, 1, 2, 3, 4, np.nan]
))

   A    B
0  x  NaN
1  x  1.0
2  x  2.0
3  y  3.0
4  y  4.0
5  y  NaN

I wanted to get the first and last rows of each group defined by column 'A'.

I tried

df.groupby('A').B.agg(['first', 'last'])

   first  last
A             
x    1.0   2.0
y    3.0   4.0

However, This doesn't give me the np.NaNs that I expected.

How do I get the actual first and last values in each group?

like image 456
piRSquared Avatar asked Aug 17 '17 20:08

piRSquared


People also ask

Can I change my order on groupby?

Groupby preserves the order of rows within each group. Unfortunately the answer to this question is NO.

Does pandas groupby keep order?

Note this does not influence the order of observations within each group. Groupby preserves the order of rows within each group.

How do you find the first value of groupby?

groupby. nth() function is used to get the value corresponding the nth row for each group. To get the first value in a group, pass 0 as an argument to the nth() function.

How do you sort values after groupby?

Sort Values in Descending Order with Groupby You can sort values in descending order by using ascending=False param to sort_values() method. The head() function is used to get the first n rows. It is useful for quickly testing if your object has the right type of data in it. Yields below output.


1 Answers

As noted here by @unutbu:

The groupby.first and groupby.last methods return the first and last non-null values respectively.

To get the actual first and last values, do:

def h(x):
    return x.values[0]

def t(x):
    return x.values[-1]

df.groupby('A').B.agg([h, t])

     h    t
A          
x  NaN  2.0
y  3.0  NaN
like image 135
piRSquared Avatar answered Sep 28 '22 17:09

piRSquared