Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

as.Date yields NA for month name "März" (march)

Tags:

date

r

xml

I got a scraped character vector with dates. My problem: When using as.Date(), every date containing the month name "März" (= which means "march" in German) is NA ed. Why is that?

Here is an (hopefully reproducible) example:

require(RCurl)
require(XML)
doc <- htmlParse(getURL("http://www.amazon.de/product-reviews/3836218984/?ie=UTF8&pageNumber=5&showViewpoints=0&sortBy=byRankDescending"), 
                 encoding="UTF-8")
(dates <- xpathSApply(doc, "//div/span[2]/nobr", xmlValue))
# [1] "12. Februar 2009"   "12. November 2006"  "19. März 2010"     
# [4] "30. Juni 2007"      "7. März 2006"       "19. März 2007"     
# [7] "22. Januar 2006"    "24. September 2005" "15. Februar 2012"  
# [10] "28. März 2007" 

Sys.setlocale("LC_TIME", "German") # on Windows, see ?Sys.setlocale
as.Date(dates,  "%d. %B %Y")
# [1] "2009-02-12" "2006-11-12" NA           "2007-06-30" NA          
# [6] NA           "2006-01-22" "2005-09-24" "2012-02-15" NA 

Any ideas on what to try next?

Note that if I apply the same on the dputed and copy/pasted character vector, everything is fine:

dates <- c("12. Februar 2009", "12. November 2006", "19. März 2010", "30. Juni 2007", 
           "7. März 2006", "19. März 2007", "22. Januar 2006", "24. September 2005", 
           "15. Februar 2012", "28. März 2007")
as.Date(dates,  "%d. %B %Y")
# [1] "2009-02-12" "2006-11-12" "2010-03-19" "2007-06-30"
# [5] "2006-03-07" "2007-03-19" "2006-01-22" "2005-09-24"
# [9] "2012-02-15" "2007-03-28"

For completeness my session info:

R version 3.0.2 (2013-09-25)
Platform: x86_64-w64-mingw32/x64 (64-bit)

locale:
[1] LC_COLLATE=German_Germany.1252  LC_CTYPE=German_Germany.1252    LC_MONETARY=German_Germany.1252
[4] LC_NUMERIC=C                    LC_TIME=German_Germany.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] tools_3.0.2
like image 676
lukeA Avatar asked Sep 07 '25 07:09

lukeA


1 Answers

I could reproduce this on Windows 7 x64. There are many issues with how R and Windows interact with character encoding, and I don't pretend to understand them. In your case, simply converting to latin1 encoding before converting to a Date should work.

as.Date(iconv(dates,from='UTF-8',to='latin1'),'%d. %B %Y')
#  [1] "2009-02-12" "2006-11-12" "2010-03-19" "2007-06-30" "2006-03-07" "2007-03-19"
#  [7] "2006-01-22" "2005-09-24" "2012-02-15" "2007-03-28"

There might be a way to get as.Date to recognize different encodings in Windows, but I don't know it.

like image 147
nograpes Avatar answered Sep 10 '25 00:09

nograpes