How to ensure R and Python scripts use the same collation?

Question

I am using R 3.6 with dplyr 1.4.2 and Python 3.7 with Pandas 1.0.3.

I need to ensure that both my R and Python scripts can sort the same vector of strings in the same order. However, in my local environment this is not the case:

In R:

library(dplyr)
df <- data.frame(
      x = c('abc(_01', 'aaa_05', 'abc_01', 'abc_01', 'abc_01', 'abc_01', NA, 'abc_01', 'abc_02', 'abc_02', 'abc_03')
)
df %>% arrange(x)

Generates:

         x
1   aaa_05
2   abc_01
3   abc_01
4   abc_01
5   abc_01
6   abc_01
7   abc_02
8   abc_02
9   abc_03
10 abc(_01
11      NA

In Python:

import pandas as pd
import numpy as np
df = pd.DataFrame({
'x': ['abc(_01', 'aaa_05', 'abc_01', 'abc_01', 'abc_01', 'abc_01', np.nan, 'abc_01', 'abc_02', 'abc_02', 'abc_03']
})
df.sort_values(['x']).reset_index(drop=True)

Generates:

          x
0    aaa_05
1   abc(_01
2    abc_01
3    abc_01
4    abc_01
5    abc_01
6    abc_01
7    abc_02
8    abc_02
9    abc_03
10      NaN

I believe this is caused by the fact that Python and R use a different collation.

If this is the case, how can I configure both languages to use the same collation?

Answer 1

To guarantee that both Python and R shells use the same collation set the same collation before creating the dataframes.

e.g.:

In R:

Sys.setlocale(category="LC_COLLATE", locale="C")
Sys.getlocale(category="LC_COLLATE")

[1] "C"

In Python:

import locale
locale.setlocale(locale.LC_COLLATE, "C")
locale.getlocale(locale.LC_COLLATE)

(None, None)