Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Counting categorical data pandas group by dataframe

Tags:

python

pandas

I have a data frame that looks like this:

+---+-----------+----------------+-------+
|   |    uid    |      msg       | count |
+---+-----------+----------------+-------+
| 0 | 121437681 | eis            |     1 |
| 1 |  14403832 | eis            |     1 |
| 2 | 190442364 | eis            |     1 |
| 3 | 190102625 | eis            |     1 |
| 4 | 190428772 | eis_reply      |     1 |
| 5 | 190428772 | single_message |     1 |
| 6 | 190428772 | yes            |     1 |
| 7 | 190104837 | eis            |     1 |
| 8 | 144969454 | eis            |     1 |
| 9 | 190738403 | eis            |     1 |
+---+-----------+----------------+-------+

What I would like to do is count the instances of each msg for each uid.

I created a groupby object and found the count of all messages:

grouped_test = test.groupby('uid')
grouped_test.count('msg') 

But I'm not quite sure how to count each type of message for each uid. I was thinking about creating masks and 4 separate data frames, but that doesn't seem like a n efficient way to accomplish this.

Sample data - http://www.sharecsv.com/s/16573757eb123c5b15cae4edcb7296e3/sample_data.csv

like image 300
metersk Avatar asked Dec 01 '22 00:12

metersk


2 Answers

Group by uid and apply value_counts to the msg column:

>>> d.groupby('uid').msg.value_counts()
uid                      
14403832   eis               1
121437681  eis               1
144969454  eis               1
190102625  eis               1
190104837  eis               1
190170637  eis               1
190428772  eis               1
           single_message    1
           yes               1
           eis_reply         1
190442364  eis               1
190738403  eis               1
190991478  single_message    1
           eis_reply         1
           yes               1
191356453  eis               1
191619393  eis               1
dtype: int64
like image 75
BrenBarn Avatar answered Dec 04 '22 08:12

BrenBarn


Apply groupby on both id and msg, and then sum the count of each:

>>> df.groupby(['uid', 'msg'])['count'].sum()
uid        msg           
14403832   eis               1
121437681  eis               1
144969454  eis               1
190102625  eis               1
190104837  eis               1
190170637  eis               1
190428772  eis               1
           eis_reply         1
           single_message    1
           yes               1
190442364  eis               1
190738403  eis               1
190991478  eis_reply         1
           single_message    1
           yes               1
191356453  eis               1
191619393  eis               1
Name: count, dtype: int64

You can reset the index to retrieve the flattened version:

>>> df.groupby(['uid', 'msg'])['count'].sum().reset_index()
          uid             msg  count
0    14403832             eis      1
1   121437681             eis      1
2   144969454             eis      1
3   190102625             eis      1
4   190104837             eis      1
5   190170637             eis      1
6   190428772             eis      1
7   190428772       eis_reply      1
8   190428772  single_message      1
9   190428772             yes      1
10  190442364             eis      1
11  190738403             eis      1
12  190991478       eis_reply      1
13  190991478  single_message      1
14  190991478             yes      1
15  191356453             eis      1
16  191619393             eis      1
like image 21
Alexander Avatar answered Dec 04 '22 08:12

Alexander