Need to read epoch time in certain timezone and then convert it to user timezone

Question

I have 2 webapps, 1 of them on cloud is "master", to which I need to match dates in the 2nd webapp "child".

Master (1st webapp, cloud) is showing date in IST, Asia/Kolkata which it reads from sql machine sitting in EST timezone.

Child (2nd webapp) reads it's data from Elasticsearch where a java feeder picks up the sql data and pushes it to Elasticsearch as it is, without any conversion.

When I try to read this Elasticsearch data in my webapp (child)

...
{
    "_index": "log_event_2016-05-05",
    "_type": "log_event",
    "_id": "65708004",
    "_score": null,
    "_source": {
        "task_name": "kn_cvs_test",
        "task_start_time": "2016-05-05T19:05:05.000-07:00",
        "task_end_time": "2016-05-05T19:05:06.000-07:00",
        "started_by": "Schedule \"10Minutes\"",
        "log_datetime": 1462475106000,
        "dw_insert_dt": "2016-05-05T16:40:54.000-07:00"
    },
    "sort": [
        1462475106000
    ]
}, {
    "_index": "log_event_2016-05-05",
    "_type": "log_event",
    "_id": "65708005",
    "_score": null,
    "_source": {
        "task_name": "kn_cvs_test",
        "task_start_time": "2016-05-05T18:55:08.000-07:00",
        "task_end_time": "2016-05-05T18:55:11.000-07:00",
        "started_by": "Schedule \"10Minutes\"",
        "log_datetime": 1462474511000,
        "dw_insert_dt": "2016-05-05T16:40:54.000-07:00"
    },
    "sort": [
        1462474511000
    ]
}
...

the dates in my webapp and the cloud does not match. Please correct me if I am wrong. Since Sql is storing dates in EST, "America/New_York", Momentjs should 1st read the data = 1462475106000 in EST and then apply the user timezone which is IST, "Asia/Kolkata". Is this correct?

//Timestamp column in table
//data  = 1462475106000
$scope.getMeData = function(data) {
    var dFormat = "YYYY-MM-DD hh:mm:ss A";
    moment.tz.setDefault("America/New_York");       
    return moment.tz(data, "Asia/Kolkata").format(dFormat); 
}

Note: 1462475106000 is the 1st entry in both table

I am putting up a plunker here. Please help me figure out what could be going wrong and how can I match dates in both the webapps (taking cloud as reference).

Update

Java feeder runs a sql query to fetch all the needed columns. Here is how log_datetime is fetched. Is this the correct way to fetch?

(task_end_time - to_date('1-1-1970 00:00:00','MM-DD-YYYY HH24:Mi:SS'))*24*3600*1000 AS "log_datetime"

So I am assuming when it fetches data Daylight saving information is not considered and I am missing out this info too. So on UI side i'll check for isDST() and do a +5:00 hrs or +4:00 hrs depending on it, as date in sql is stored in America/New_York. Plunker with UI fix

Answer 1

There's only one Epoch time anywhere at a time. It's an absolute value.

A value that approximates the number of seconds that have elapsed since the Epoch. A Coordinated Universal Time name (specified in terms of seconds (tm_sec), minutes (tm_min), hours (tm_hour), days since January 1 of the year (tm_yday), and calendar year minus 1900 (tm_year)) is related to a time represented as seconds since the Epoch, according to the expression below.

Ref http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_15

Epoch time is measured from the 01-01-1970 00:00:00 UTC. That's an absolute instant in time, it's not relative to timezone as the timezone UTC is in definition of the epoch.

Whenever you see a number for a date such as your example 1462475106000, it should be assumed to be in UTC/GMT. It's not a timezone specific value. Since the time is Unix based number, by epoch definition it is time passed since epoch (in UTC), making it absolute time.

(01-01-1970 00:00:00 UTC)(epoch) + 1462475106000 = Time in UTC

A date in string is a different matter entirely. It should include the timezone offset with the date part. Date part would be considered to be timezone specific value. Adding timezone offset to it would convert it to absolute value in UTC. Without timezone information a string date has no absolute value (Point in time).

2016-05-05T18:55:08.000-07:00 = (2016-05-05T18:55:08.000) + (07:00 hrs) UTC
or
date + timezone offset = date - offset hrs UTC

Databases don't store dates in any timzzone. They store absolute Unix time value. Timezone setting in databases are set so that the output of the query in string is shown in that timezone format. This setting only specifies the output date format not the value. Since the value is absolute it's same timezone independent.

This setting also helps database in determining the timezone of the date value that a user inserts if he misses the timezone offset. If the user tries to insert a string in date column without timezone database would try to default the string to database timezone setting.

So 1462475106000 is a value in UTC as it's Unix time. It's not in EST.

If you need date to be in EST then use the date in string format not in number format as number format is always UTC.

moment.tz.setDefault("America/New_York");//sets the output date format in EST

Above code will have no effect as it's overwritten by moment.tz(data, "Asia/Kolkata")

Now considering the first entry in the first screenshot.

2016-05-06T04:35:06.000+5:30 (IST) = 2016-05-05T11:05:06.000-00:00 (UTC)

Comparing it with the second screenshot, since the time difference between the two is 4 hours, second one should be in +01:30 timezone, if they both are the same values in absolute time.

2016-05-06T12:35:06.000+1:30 = 2016-05-05T11:05:06.000-00:00 (UTC)

However in the child the value is

2016-05-05T19:05:06.000-07:00 = 2016-05-06T02:05:06.000-00:00 (UTC)

Which is not the same value as both above. Interesting fact is that only date part in the child 2016-05-05T19:05:06.000 when added IST offset +05:30would become the date part in second screenshot.

2016-05-06T12:35:06.00 - 2016-05-05T19:05:06.000 = 5:30 (IST offset)

Leaving that aside your Java feeder code to ElasticSearch is probably culprit here.

It's not writing the correct date-timezone combination to ElasticSearch. Try using the overloaded method getDate() getDate(int columnIndex, Calendar cal) in JDBC.

Sometimes JDBC driver misses timezone information, causing the date to be stored in default database timezone, using Calendar would cure that.

Same goes for writing dates as well.

void setDate(int parameterIndex, Date x, Calendar cal) throws SQLException