Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to replace a pure-number column by a number-keyword dict ? [python]

I have a dataframe and a dict below, but how do i replace the column by the dict?

data
index     occupation_code
0          10
1          16
2          12
3           7
4           1
5           3
6          10
7           7
8           1
9           3
10          4
……

dict1 = {0: 'other',1: 'academic/educator',2: 'artist',3: 'clerical/admin',4: 'college/grad student',5: 'customer service',6: 'doctor/health care',7: 'executive/managerial',8: 'farmer',9: 'homemaker',10: 'K-12student',11: 'lawyer',12: 'programmer',13: 'retired',14: 'sales/marketing',15: 'scientist',16: 'self-employed',17: 'technician/engineer',18: 'tradesman/craftsman',19: 'unemployed',20: 'writer'}

I used a "for"sentence to do the replace, but it's very slow, like that:

for i in data.index:
  data.loc[i,'occupation_detailed'] = dict1[data.loc[i,'occupation_code']]

Since my data contains 1 million lines and it costs several seconds if i only run it for 1 thousand times. 1 million line may cost a half day!

So Is there any better way to do that?

Great thanks for ur suggestions!

like image 763
Ricky Avatar asked Apr 22 '17 14:04

Ricky


2 Answers

Use map and if some value missing get NaN:

print (df)
       occupation_code
index                 
0                   10
1                   16
2                   12
3                    7
4                    1
5                    3
6                   10
7                    7
8                    1
9                    3
10                   4
11                 100 <- add missing value 100

df['occupation_code'] = df['occupation_code'].map(dict1)
print (df)
            occupation_code
index                      
0               K-12student
1             self-employed
2                programmer
3      executive/managerial
4         academic/educator
5            clerical/admin
6               K-12student
7      executive/managerial
8         academic/educator
9            clerical/admin
10     college/grad student
11                      NaN

Another solution is use replace, if some values missing get original value, no NaN:

df['occupation_code'] = df['occupation_code'].replace(dict1)
print (df)
            occupation_code
index                      
0               K-12student
1             self-employed
2                programmer
3      executive/managerial
4         academic/educator
5            clerical/admin
6               K-12student
7      executive/managerial
8         academic/educator
9            clerical/admin
10     college/grad student
11                      100
like image 137
jezrael Avatar answered Oct 04 '22 07:10

jezrael


Assuming @jezrael's sample data df

print(df)

       occupation_code
index                 
0                   10
1                   16
2                   12
3                    7
4                    1
5                    3
6                   10
7                    7
8                    1
9                    3
10                   4
11                 100

I'd recommend using the get method of a dictionary embedded in a lambda. This allows you to embed a default value for things not in the dictionary. In this case, I return the original value.

df.occupation_code.map(lambda x: dict1.get(x, x))

index
0              K-12student
1            self-employed
2               programmer
3     executive/managerial
4        academic/educator
5           clerical/admin
6              K-12student
7     executive/managerial
8        academic/educator
9           clerical/admin
10    college/grad student
11                     100
Name: occupation_code, dtype: object
like image 25
piRSquared Avatar answered Oct 04 '22 07:10

piRSquared