Grouping data by value ranges

Question

I have a csv file that shows parts on order. The columns include days late, qty and commodity.

I need to group the data by days late and commodity with a sum of the qty. However the days late needs to be grouped into ranges.

>56
>35 and <= 56
>14 and <= 35
>0 and <=14

I was hoping I could use a dict some how. Something like this

{'Red':'>56,'Amber':'>35 and <= 56','Yellow':'>14 and <= 35','White':'>0 and <=14'}

I am looking for a result like this

        Red  Amber  Yellow  White
STRSUB  56   60     74      40
BOTDWG  20   67     87      34

I am new to pandas so I don't know if this is possible at all. Could anyone provide some advice.

Thanks

Answer 1

Suppose you start with this data:

df = pd.DataFrame({'ID': ('STRSUB BOTDWG'.split())*4,
                   'Days Late': [60, 60, 50, 50, 20, 20, 10, 10],
                   'quantity': [56, 20, 60, 67, 74, 87, 40, 34]})
#    Days Late      ID  quantity
# 0         60  STRSUB        56
# 1         60  BOTDWG        20
# 2         50  STRSUB        60
# 3         50  BOTDWG        67
# 4         20  STRSUB        74
# 5         20  BOTDWG        87
# 6         10  STRSUB        40
# 7         10  BOTDWG        34

Then you can find the status category using pd.cut. Note that by default, pd.cut splits the Series df['Days Late'] into categories which are half-open intervals, (-1, 14], (14, 35], (35, 56], (56, 365]:

df['status'] = pd.cut(df['Days Late'], bins=[-1, 14, 35, 56, 365], labels=False)
labels = np.array('White Yellow Amber Red'.split())
df['status'] = labels[df['status']]
del df['Days Late']
print(df)
#        ID  quantity  status
# 0  STRSUB        56     Red
# 1  BOTDWG        20     Red
# 2  STRSUB        60   Amber
# 3  BOTDWG        67   Amber
# 4  STRSUB        74  Yellow
# 5  BOTDWG        87  Yellow
# 6  STRSUB        40   White
# 7  BOTDWG        34   White

Now use pivot to get the DataFrame in the desired form:

df = df.pivot(index='ID', columns='status', values='quantity')

and use reindex to obtain the desired order for the rows and columns:

df = df.reindex(columns=labels[::-1], index=df.index[::-1])

---

Thus,

import numpy as np
import pandas as pd

df = pd.DataFrame({'ID': ('STRSUB BOTDWG'.split())*4,
                   'Days Late': [60, 60, 50, 50, 20, 20, 10, 10],
                   'quantity': [56, 20, 60, 67, 74, 87, 40, 34]})
df['status'] = pd.cut(df['Days Late'], bins=[-1, 14, 35, 56, 365], labels=False)
labels = np.array('White Yellow Amber Red'.split())
df['status'] = labels[df['status']]
del df['Days Late']
df = df.pivot(index='ID', columns='status', values='quantity')
df = df.reindex(columns=labels[::-1], index=df.index[::-1])
print(df)

yields

        Red  Amber  Yellow  White
ID                               
STRSUB   56     60      74     40
BOTDWG   20     67      87     34

Answer 2

You can create a column in your DataFrame based on your Days Late column by using the map or apply functions as follows. Let's first create some sample data.

df = pandas.DataFrame({ 'ID': 'foo,bar,foo,bar,foo,bar,foo,foo'.split(','),
                        'Days Late': numpy.random.randn(8)*20+30})

   Days Late   ID
0  30.746244  foo
1  16.234267  bar
2  14.771567  foo
3  33.211626  bar
4   3.497118  foo
5  52.482879  bar
6  11.695231  foo
7  47.350269  foo

Create a helper function to transform the data of the Days Late column and add a column called Code.

def days_late_xform(dl):
    if dl > 56: return 'Red'
    elif 35 < dl <= 56: return 'Amber'
    elif 14 < dl <= 35: return 'Yellow'
    elif 0 < dl <= 14: return 'White'
    else: return 'None'

df["Code"] = df['Days Late'].map(days_late_xform)

   Days Late   ID    Code
0  30.746244  foo  Yellow
1  16.234267  bar  Yellow
2  14.771567  foo  Yellow
3  33.211626  bar  Yellow
4   3.497118  foo   White
5  52.482879  bar   Amber
6  11.695231  foo   White
7  47.350269  foo   Amber

Lastly, you can use groupby to aggregate by the ID and Code columns, and get the counts of the groups as follows:

g = df.groupby(["ID","Code"]).size()
print g

ID   Code
bar  Amber     1
     Yellow    2
foo  Amber     1
     White     2     
     Yellow    2

df2 = g.unstack()
print df2

Code  Amber  White  Yellow
ID
bar       1    NaN       2
foo       1      2       2