I'm trying to create what I think is a simple pivot table but am having serious issues. There are two things I'm unable to do:
Setup:
df = pd.DataFrame({'company':['a','b','c','b'], 'partner':['x','x','y','y'], 'str':['just','some','random','words']})
Desired Output:
company x y
a True False
b True True
c False True
I started with:
df = df.pivot(values = 'partner', columns = 'partner', index = 'company').reset_index()
which gets me close, but when I try to get rid of the "partner" column, I can't even reference it, and it's not the "index".
For the second issue, I can use:
df.fillna(False, inplace = True)
df.loc[~(df['x'] == False), 'x'] = True
df.loc[~(df['y'] == False), 'y'] = True
but that seems incredibly hacky. Any help would be appreciated.
Option 1
df.groupby(['company', 'partner']).size().unstack(fill_value=0).astype(bool)
partner x y
company
a True False
b True True
c False True
Get rid of names on columns object
df.groupby(['company', 'partner']).size().unstack(fill_value=0).astype(bool) \
.rename_axis(None, 1).reset_index()
company x y
0 a True False
1 b True True
2 c False True
Option 2
pd.crosstab(df.company, df.partner).astype(bool)
partner x y
company
a True False
b True True
c False True
pd.crosstab(df.company, df.partner).astype(bool) \
.rename_axis(None, 1).reset_index()
company x y
0 a True False
1 b True True
2 c False True
Option 3
f1, u1 = pd.factorize(df.company.values)
f2, u2 = pd.factorize(df.partner.values)
n, m = u1.size, u2.size
b = np.bincount(f1 * m + f2)
pad = np.zeros(n * m - b.size, dtype=int)
b = np.append(b, pad)
v = b.reshape(n, m).astype(bool)
pd.DataFrame(np.column_stack([u1, v]), columns=np.append('company', u2))
company x y
0 a True False
1 b True True
2 c False True
Timing
small data
%timeit df.groupby(['company', 'partner']).size().unstack(fill_value=0).astype(bool).rename_axis(None, 1).reset_index()
%timeit pd.crosstab(df.company, df.partner).astype(bool).rename_axis(None, 1).reset_index()
%%timeit
f1, u1 = pd.factorize(df.company.values)
f2, u2 = pd.factorize(df.partner.values)
n, m = u1.size, u2.size
b = np.bincount(f1 * m + f2)
pad = np.zeros(n * m - b.size, dtype=int)
b = np.append(b, pad)
v = b.reshape(n, m).astype(bool)
pd.DataFrame(np.column_stack([u1, v]), columns=np.append('company', u2))
1000 loops, best of 3: 1.67 ms per loop
100 loops, best of 3: 5.97 ms per loop
1000 loops, best of 3: 301 µs per loop
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With