Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas: Pivot to True/False, drop column

I'm trying to create what I think is a simple pivot table but am having serious issues. There are two things I'm unable to do:

  1. Get rid of the "partner" column at the end.
  2. Set the values to either True or False if each company has that partner.

Setup:

df = pd.DataFrame({'company':['a','b','c','b'], 'partner':['x','x','y','y'], 'str':['just','some','random','words']})

Desired Output:

company     x      y  
a        True  False
b        True   True
c       False   True

I started with:

df = df.pivot(values = 'partner', columns = 'partner', index = 'company').reset_index()

which gets me close, but when I try to get rid of the "partner" column, I can't even reference it, and it's not the "index".

For the second issue, I can use:

df.fillna(False, inplace = True)
df.loc[~(df['x'] == False), 'x'] = True
df.loc[~(df['y'] == False), 'y'] = True

but that seems incredibly hacky. Any help would be appreciated.

like image 393
elPastor Avatar asked Jan 04 '23 22:01

elPastor


1 Answers

Option 1

df.groupby(['company', 'partner']).size().unstack(fill_value=0).astype(bool)


partner      x      y
company              
a         True  False
b         True   True
c        False   True

Get rid of names on columns object

df.groupby(['company', 'partner']).size().unstack(fill_value=0).astype(bool) \
    .rename_axis(None, 1).reset_index()

  company      x      y
0       a   True  False
1       b   True   True
2       c  False   True

Option 2

pd.crosstab(df.company, df.partner).astype(bool)

partner      x      y
company              
a         True  False
b         True   True
c        False   True


pd.crosstab(df.company, df.partner).astype(bool) \
    .rename_axis(None, 1).reset_index()

  company      x      y
0       a   True  False
1       b   True   True
2       c  False   True

Option 3

f1, u1 = pd.factorize(df.company.values)
f2, u2 = pd.factorize(df.partner.values)
n, m = u1.size, u2.size

b = np.bincount(f1 * m + f2)
pad = np.zeros(n * m - b.size, dtype=int)
b = np.append(b, pad)
v = b.reshape(n, m).astype(bool)

pd.DataFrame(np.column_stack([u1, v]), columns=np.append('company', u2))

  company      x      y
0       a   True  False
1       b   True   True
2       c  False   True

Timing
small data

%timeit df.groupby(['company', 'partner']).size().unstack(fill_value=0).astype(bool).rename_axis(None, 1).reset_index()
%timeit pd.crosstab(df.company, df.partner).astype(bool).rename_axis(None, 1).reset_index()

%%timeit
f1, u1 = pd.factorize(df.company.values)
f2, u2 = pd.factorize(df.partner.values)
n, m = u1.size, u2.size

b = np.bincount(f1 * m + f2)
pad = np.zeros(n * m - b.size, dtype=int)
b = np.append(b, pad)
v = b.reshape(n, m).astype(bool)

pd.DataFrame(np.column_stack([u1, v]), columns=np.append('company', u2))

1000 loops, best of 3: 1.67 ms per loop
100 loops, best of 3: 5.97 ms per loop
1000 loops, best of 3: 301 µs per loop
like image 111
piRSquared Avatar answered Jan 16 '23 20:01

piRSquared