Transform multiple categorical columns

Question

In my dataset I have two categorical columns which I would like to numerate. The two columns both contain countries, some overlap (appear in both columns). I would like to give the same number in column1 and column2 for the same country.

My data looks somewhat like:

import pandas as pd

d = {'col1': ['NL', 'BE', 'FR', 'BE'], 'col2': ['BE', 'NL', 'ES', 'ES']}
df = pd.DataFrame(data=d)
df

Currenty I am transforming the data like:

from sklearn.preprocessing import LabelEncoder
df.apply(LabelEncoder().fit_transform)

However this makes no distinction between FR and ES. Is there another simple way to come to the following output?

o = {'col1': [2,0,1,0], 'col2': [0,2,4,4]}
output = pd.DataFrame(data=o)
output

Answer 1

Here is one way

df.stack().astype('category').cat.codes.unstack()
Out[190]: 
   col1  col2
0     3     0
1     0     3
2     2     1
3     0     1

Or

s=df.stack()
s[:]=s.factorize()[0]
s.unstack()
Out[196]: 
   col1  col2
0     0     1
1     1     0
2     2     3
3     1     3

Answer 2

You can fit the LabelEncoder() with the unique values in your dataframe first and then transform.

le = LabelEncoder()
le.fit(pd.concat([df.col1, df.col2]).unique()) # or np.unique(df.values.reshape(-1,1))

df.apply(le.transform)
Out[28]: 
   col1  col2
0     3     0
1     0     3
2     2     1
3     0     1