Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get the Weight of Evidence (WOE) and Information Value (IV) in Python/pandas?

I was wondering how to calculate the WOE and IV in python. Are there any dedication function in numpy/scipy/pandas/sklearn?

Here is my example dataframe:

import numpy as np
import pandas as pd
np.random.seed(100)


df = pd.DataFrame({'grade': np.random.choice(list('ABCD'),size=(20)),
                   'pass': np.random.choice([0,1],size=(20))
})
df
like image 784
BhishanPoudel Avatar asked Dec 18 '22 13:12

BhishanPoudel


1 Answers

Formulas for woe and iv:

enter image description here

Code to achieve this:

import numpy as np
import pandas as pd
np.random.seed(100)


df = pd.DataFrame({'grade': np.random.choice(list('ABCD'),size=(20)),
                   'pass': np.random.choice([0,1],size=(20))
})

feature,target = 'grade','pass'
df_woe_iv = (pd.crosstab(df[feature],df[target],
                      normalize='columns')
             .assign(woe=lambda dfx: np.log(dfx[1] / dfx[0]))
             .assign(iv=lambda dfx: np.sum(dfx['woe']*
                                           (dfx[1]-dfx[0]))))

df_woe_iv

output

pass     0    1       woe        iv
grade                              
A      0.3  0.3  0.000000  0.690776
B      0.1  0.1  0.000000  0.690776
C      0.2  0.5  0.916291  0.690776
D      0.4  0.1 -1.386294  0.690776

like image 159
BhishanPoudel Avatar answered Apr 28 '23 10:04

BhishanPoudel