Handling time zone offset transition and daylight savings time with Joda

Question

I am trying to parse datetime strings and create Joda DateTime objects.

My data comes from a legacy database that stores datetime strings without specifying the timezone/offset. Although the timezone/offset of the datetime strings is not stored it is a business rule of the legacy system that all datetimes are stored in Eastern Time. Unfortunately I do not have the authority to update the way in which the legacy DB stores datetime strings.

Thus, I parse the datetime strings using JODA's "US/Eastern" time zone.

This approach throws an illegalInstance exception when the dateTime string falls within the hour that "disappears" when daylight savings is switched on.

I've created the following example code to demonstrate this behaviour and to show my proposed workaround.

public class FooBar {
public static final DateTimeZone EST = DateTimeZone.forID("EST");
public static final DateTimeZone EASTERN = DateTimeZone.forID("US/Eastern");

public static final DateTimeFormatter EST_FORMATTER = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSS").withZone(EST);
public static final DateTimeFormatter EASTERN_FORMATTER = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSS").withZone(EASTERN);


public static void main(String[] args) {
    final String[] listOfDateTimeStrings = {"2014-03-09 02:00:00.000", "2014-03-08 02:00:00.000"}; 

    System.out.println(" *********** 1st attempt  *********** ");
    for (String dateTimeString: listOfDateTimeStrings){
        try{
            final DateTime dateTime = DateTime.parse(dateTimeString, EASTERN_FORMATTER);
            System.out.println(dateTime);       
        }
        catch(Exception e){
            System.out.println(e.getMessage());
        }
    }

    System.out.println(" *********** 2nd attempt  *********** ");
    for (String dateTimeString: listOfDateTimeStrings){
        try{
            final DateTime dateTime = DateTime.parse(dateTimeString, EST_FORMATTER);
            System.out.println(dateTime);       
        }
        catch(Exception e){
            System.out.println(e.getMessage());
        }
    }

    System.out.println(" *********** 3rd attempt  *********** ");
    for (String dateTimeString: listOfDateTimeStrings){
        try{
            DateTime dateTime = DateTime.parse(dateTimeString, EST_FORMATTER);
            dateTime = dateTime.withZone(EASTERN);
            System.out.println(dateTime);       
        }
        catch(Exception e){
            System.out.println(e.getMessage());
        }
    }       

}

}

Output produced:

 *********** 1st attempt  *********** 
Cannot parse "2014-03-09 02:00:00.000": Illegal instant due to time zone offset transition (America/New_York)
2014-03-08T02:00:00.000-05:00
 *********** 2nd attempt  *********** 
2014-03-09T02:00:00.000-05:00
2014-03-08T02:00:00.000-05:00
 *********** 3rd attempt  *********** 
2014-03-09T03:00:00.000-04:00
2014-03-08T02:00:00.000-05:00

In the "3rd attempt" I get the expected result: the first datetime has an offset of -04:00. as it falls within the first hour of DST for 2015. The second timestamp has an offset of -05:00 as it falls outside of DST.

Is is safe to do this:

DateTime dateTime = DateTime.parse(dateTimeString, A_FORMATTER_WITH_TIME_ZONE_A);
dateTime = dateTime.withZone(TIME_ZONE_B);

I've tested this code with a few different combinations of datetime strings and time zones (and so far it works for all the test cases), but I was wondering if anyone with more Joda experience can see anything wrong/dangerous in this approach.

Or alternatively: is there a better way of doing handling the time zone offset transition with Joda?

Answer 1

Be careful. The behaviour of the method withZone(...) is documented as follows:

Returns a copy of this datetime with a different time zone, preserving the millisecond instant.

Keeping this in mind, you have to understand that EST and "America/New_York" (better than the outdated id "US/Eastern") are not the same. First one (EST) has fixed offset (no DST), but second one has DST including possible gaps. You should only apply EST as replacement for Eastern if you are sure that

a) you are already in the exception mode (normally parsed datetimes in Eastern zone should be accepted without repeated parsing otherwise applying EST would falsify the parsed instant),

b) you understand choosing EST in second (and third) try is choosing the instant after DST transition.

Regarding this limitations/constraints, your workaround will work (but only for the special pair EST versus America/New_York). Personally I find it frightening to use exception-based logic to workaround a severe limitation of Joda-Time. As counter example, the new JSR-310 does not use the exception strategy when handling gaps but the strategy to choose the later instant after gap pushed forward by the size of the gap (like the old java.util.Calendar-stuff).

I recommend you to first follow the advise of @Jim Garrison to look if the crapped data can be corrected before you apply such a workaround (my upvote for his answer).

Update after reading your original spec requirement (obsolete - see below):

If the spec of the legacy system says that all times are stored in EST, then you should parse it this way and NOT use "America/New_York" for parsing at all. Instead you can convert the parsed EST-instants to New-York-time in second phase (using withZone(EASTERN). This way you will not have any exception logic because a (parsed) instant can always be converted to a local time representation in an unambigous way (the parsed instant of type DateTime, the converted result contains a local time). Code example:

public static final DateTimeZone EST = DateTimeZone.forID("EST");
public static final DateTimeZone EASTERN = DateTimeZone.forID("US/Eastern");
public static final DateTimeFormatter EST_FORMATTER = 
  DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSS").withZone(EST);

// in your parsing method...
String input = "2014-03-09 02:00:00.000";
DateTime dt = EST_FORMATTER.parseDateTime(input);
System.out.println(dt); // 2014-03-09T02:00:00.000-05:00
System.out.println(dt.withZone(EASTERN)); // 2014-03-09T03:00:00.000-04:00

Update after comment and clarification of OP:

Now it is confirmed that the legacy system does not store timestamps in EST (with fixed offset UTC-05 but in EASTERN zone ("America/New_York" with variable offset of either EST or EDT). First action should be to contact the supplier of invalid timestamps in order to see if they can correct the data. Else you can use following workaround:

Regarding the fact that your input contains timestamps without any offset or zone information I recommend first to parse as LocalDateTime.

=> static initialization part

// Joda-Time cannot parse "EDT" so we use hard-coded offsets
public static final DateTimeZone EST = DateTimeZone.forOffsetHours(-5);
public static final DateTimeZone EDT = DateTimeZone.forOffsetHours(-4);

public static final DateTimeZone EASTERN = DateTimeZone.forID("America/New_York");
public static final org.joda.time.format.DateTimeFormatter FORMATTER = 
    org.joda.time.format.DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss.SSS");

=> in your parse method

String input = "2014-03-09 02:00:00.000";
LocalDateTime ldt = FORMATTER.parseLocalDateTime(input); // always working
System.out.println(ldt); // 2014-03-09T02:00:00.000
DateTime result;

try {
    result = ldt.toDateTime(EASTERN);
} catch (IllegalInstantException ex) {
    result = ldt.plusHours(1).toDateTime(EDT); // simulates a PUSH-FORWARD-strategy at gap
    // result = ldt.toDateTime(EST); // the same instant but finally display with EST offset
}
System.out.println(result); // 2014-03-09T03:00:00.000-04:00
// if you had chosen <<<ldt.toDateTime(EST)>>> then: 2014-03-09T02:00:00.000-05:00

Another clarification due to last comment of OP:

The method toDateTime(DateTimeZone) producing a DateTime is documented as follows:

In a daylight saving overlap, when the same local time occurs twice, this method returns the first occurrence of the local time.

In other words, it chooses the earlier offset in case of overlap (in autumn). So there is no need to call

result = ldt.toDateTime(EASTERN).withEarlierOffsetAtOverlap();

However, it does not do any harm here, and you might prefer it for sake of documentation. On the other side: It does not make any sense to call in the exception handling (for gaps)

result = ldt.toDateTime(EDT).withEarlierOffsetAtOverlap();

because EDT (and EST, too) is a fixed offset where overlaps can never occur. So here the method withEarlierOffsetAtOverlap() does not do anything. Furthermore: Leaving out the correction ldt.plusHours(1) in case of EDT is not okay and will yield another instant. Already tested by me before writing this extra explanation, but of course, you can use the alternative ldt.toDateTime(EST) to achieve what you want (EDT != EST, but with the correction of plusHours(1) you get same instants). I have just noticed the EDT example to demonstrate how you can exactly model the standard JDK-behaviour. It is up to you which offset you prefer in case of resolving gaps (EDT or EST) but getting the same instants is vital here (ldt.plusHours(1).toDateTime(EDT) versus result = ldt.toDateTime(EST)).

Answer 2

Your question boils down to

I have to cope with timestamp values that are not technically valid and interpret them consistently

What you do with them depends totally on your requirements. If you are writing the software for an employer, the owner of the project must be the one to make that decision. You as the developer do not yet have the authority to decide what to do with invalid input if that hasn't been specified by the designer/architect.

I suggest you go back to the project owner/manager, inform them of the problem (input contains date/time stamps that don't actually exist) and get them to make the decision on how to handle them.