Dealing with Eastern Standard Time (EST) and Eastern Daylight Savings (EDT) in R

Question

I have a series of datasets from a water quality continuous monitoring probe with the Date and Time set in Eastern Standard Time (EST) so there is no correction of Daylight Savings Time (EDT). In R, the fields are recognized as factors when importing the data table from an MS access database however when converting using as.POSIXct() dates and times from 02:00 (24 clock) on 2016-03-13 become NAs. This is due to the transition from EST to EDT...therefore technically 2016-03-13 02:00 doesn't exist.

Some created data as an example

test<-data.frame(Date=rep(as.Date("2016-03-13"),120),Hour=rep(seq(0,23,1),5),Min=rep(seq(0,60,15),24))

Is there a way to convert the factor or character field to as POSIXct field while retaining the EST timezone designation? Alternatively is there a way to identify and convert the proper date and times into EST and EDT?

I have gone around and around and can't get anything to work. I have attempted to convert to GMT (or UTC) then convert back to EST (tz="America/New_York"). I realize that this is an ongoing issue and people who work with date and time data, especially in R would love to move away from EDT.

Any help is appreciated...I am at my wits end on this one.

Answer 1

The trouble with using POSIX tz = "America/New_York" is that daylight savings time is accounted for (UTC-4 or UTC-5), even if the underlying timestamps are stored in U.S. Eastern Standard Time (UTC-5).

You should be able to designate your tz as Etc/GMT+5. From there, it is easy to convert between EST, Eastern local time, and GMT. Note that in R, time zones west of UTC are denoted with a positive offset (see Time Zone Names documentation in ?timezone).

Here is some example data (daylight savings took effect at 2:00 AM Eastern Time on 3/16/16):

StartTime=as.numeric(as.POSIXct("2016-03-11 0:00:00",format="%Y-%m-%d %H:%M",origin="1970-01-01",tz="Etc/GMT+5"))
EndTime=as.numeric(as.POSIXct("2016-03-15 0:00:00",format="%Y-%m-%d %H:%M",origin="1970-01-01",tz="Etc/GMT+5"))

Interval=15*60  #15-min
data.EST=as.POSIXct(seq(from = StartTime,to = EndTime, by=Interval),origin="1970-01-01",tz="Etc/GMT+5") #generate date stamps

# convert Eastern Standard Time (in R: GMT+5) to local time (accounts for daylight savings):
data.EastCoast<- format(data.EST, tz="America/New_York")

# convert Eastern Standard Time (in R: GMT+5) to UTC/GMT:
data.UTC<- format(data.EST, tz="GMT")

compare.times<-data.frame(data.EST,data.EastCoast,data.UTC)

compare.times[(198:203),]

               data.EST      data.EastCoast            data.UTC
198 2016-03-13 01:15:00   2016-03-13 01:15:00   2016-03-13 06:15:00
199 2016-03-13 01:30:00   2016-03-13 01:30:00   2016-03-13 06:30:00
200 2016-03-13 01:45:00   2016-03-13 01:45:00   2016-03-13 06:45:00
201 2016-03-13 02:00:00   2016-03-13 03:00:00   2016-03-13 07:00:00
202 2016-03-13 02:15:00   2016-03-13 03:15:00   2016-03-13 07:15:00
203 2016-03-13 02:30:00   2016-03-13 03:30:00   2016-03-13 07:30:00

Good luck!

Answer 2

During the conversion to POSIX you need to specify the time zone. See this example:

test<-data.frame(Date=rep(as.Date("2016-03-13"),96),Hour=rep(seq(0,23,1), each=4),Min=rep(seq(0,45,15)))
wrong<-as.POSIXct(paste(test$Date, test$Hour, test$Min), format="%Y-%m-%d %H %M")
ans<-as.POSIXct(paste(test$Date, test$Hour, test$Min), format="%Y-%m-%d %H %M", tz="EST")

compare<-cbind(test, wrong, ans)

In the vector "wrong", no timezone was specified thus the NA, but in the second case Eastern Standard was specified and the desired result is given.