Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas set_levels on MultiIndex: Level values must be unique

Given a DataFrame df

                    Value
Category Pool Class      
A        1.0  1.0       1
              9.0       2
B        1.0  1.0       3
C        1.0  1.0       4
              5.0       5

I want to convert the levels Pool and Class to integers without reset_index (see below).

I tried using a combination of get_level_values and set_levels like so

for c in ['Pool', 'Class']:
    df.index.set_levels(df.index.get_level_values(c).astype(int), level=c, inplace=True)

However, this raises

ValueError: Level values must be unique: [1, 1, 1, 1, 1] on level 1

To understand what happens, I also tried using verify_integrity=False. Then

df.index.set_levels(df.index.get_level_values('Class').astype(int),
                    level='Class', verify_integrity=False, inplace=True)

produces

                    Value
Category Pool Class      
A        1.0  1         1
              1         2
B        1.0  1         3
C        1.0  1         4
              9         5

whereas my goal is to obtain

                    Value
Category Pool Class      
A        1.0  1         1
              9         2
B        1.0  1         3
C        1.0  1         4
              5         5

How to achieve this properly? Is chaining of get_level_values and set_levels the correct way to do it? Why is pandas not able to properly set the level after having it transformed with astype?

I guess you could work with reset_index and set_index but what is the benefit then of having the methods set_levels?

d = {'Category': str, 'Pool': int, 'Class': int}
df.reset_index(drop=False, inplace=True)
for k, v in d.items():
    df[k] = df[k].astype(v)

df.set_index(list(d.keys()), inplace=True)
like image 805
gofvonx Avatar asked Jan 28 '23 06:01

gofvonx


1 Answers

You can access index levels directly via pd.MultiIndex.levels and feed to pd.MultiIndex.set_levels:

df.index = df.index.set_levels(df.index.levels[2].astype(int), level=2)

print(df)

                     Value
Category Pool Class       
A        1.0  1          1
              9          2
B        1.0  1          3
C        1.0  1          4
              5          5
like image 148
jpp Avatar answered Jan 31 '23 09:01

jpp