How to open dbf file in UTF-8

Question

I have a dbf file in UTF-8 encoding. When I open it in R, with read.dbf() (package foreign) on Windows with CP1250 charset, characters are damaged:

require(foreign)
x <- read.dbf('file.dbf')
Sys.setlocale()
# "LC_COLLATE=Czech_Czech Republic.1250;LC_CTYPE=Czech_Czech Republic.1250;LC_MONETARY=Czech_Czech Republic.1250;LC_NUMERIC=C;LC_TIME=Czech_Czech Republic.1250"

I was wondering whether I should set different locale, as advised on some answers here, but it doesn't work on Win XP - when I try to set the locale to UTF-8, it says "OS reports request to set locale to "Czech_Czech Rebublic.UTF8" cannot be honored".

I don't think that setting a new locale should be necessary by design (also see here) - it should be the function read.dbf() probably, or maybe some postprocessing, to handle the charset without having to switch the locale of the whole R system.

Do you know how to do that?

Answer 1

I have a similar problem, working with Slovak characters.

I read the table using read.dbf, while I set the as.is = TRUE, to read the characters as characters, not as factors. Once the file is in R, I specify the Encoding to UTF-8 using Encoding(tab$NAM) <-"UTF-8".

This is how it looks like:

require(foreign)

# set slovak characters
Sys.setlocale(category = "LC_ALL", locale = "Slovak")

# Read data in .dbf (Usually exported from ArcGIS 10.4)
tab<- read.dbf("C:/Users/tab.dbf", as.is = TRUE)

# Check how special characters looks like
tab$NAM
"Veľký Polom"                 "Malý Polom"  "Veľká Rača"                 
"Fabova hoÄľa" 

# Set the correct encoding for your column with special characters
Encoding(tab$NAM) <-"UTF-8"

# Check it again
tab$NAM
"Veľký Polom"                "Malý Polom"                 "Veľká Rača"                 
"Fabova hoľa"                 

Voila!