Python 3: timestamp to datetime: where does this additional hour come from?

Question

I'm using the following functions:

# The epoch used in the datetime API.
EPOCH = datetime.datetime.fromtimestamp(0)

def timedelta_to_seconds(delta):
    seconds = (delta.microseconds * 1e6) + delta.seconds + (delta.days * 86400)
    seconds = abs(seconds)

    return seconds

def datetime_to_timestamp(date, epoch=EPOCH):
    # Ensure we deal with `datetime`s.
    date = datetime.datetime.fromordinal(date.toordinal())
    epoch = datetime.datetime.fromordinal(epoch.toordinal())

    timedelta = date - epoch
    timestamp = timedelta_to_seconds(timedelta)

    return timestamp

def timestamp_to_datetime(timestamp, epoch=EPOCH):
    # Ensure we deal with a `datetime`.
    epoch = datetime.datetime.fromordinal(epoch.toordinal())

    epoch_difference = timedelta_to_seconds(epoch - EPOCH)
    adjusted_timestamp = timestamp - epoch_difference

    date = datetime.datetime.fromtimestamp(adjusted_timestamp)

    return date

And using them with the passed code:

twenty = datetime.datetime(2010, 4, 4)

print(twenty)
print(datetime_to_timestamp(twenty))
print(timestamp_to_datetime(datetime_to_timestamp(twenty)))

And getting the following results:

2010-04-04 00:00:00
1270339200.0
2010-04-04 01:00:00

For some reason, I'm getting an additional hour added in the last call, despite my code having, as far as I can see, no flaws.

Where is this additional hour coming from?

Answer 1

# Ensure we deal with `datetime`s.
date = datetime.datetime.fromordinal(date.toordinal())

(That's chopping off the time-of-day completely, as ‘ordinal’ is only a day number. Is that what you meant to do? I suspect not.)

Anyway, as Michael said, datetime.fromtimestamp gives you a naïve datetime corresponding to what local time for that POSIX (UTC) timestamp would be for you. So when you call —

date = datetime.datetime.fromtimestamp(adjusted_timestamp)

you're getting the local time for the POSIX timestamp representing 2010-04-04T00:00:00, which of course in BST is an hour ahead. This doesn't happen in the return direction because your epoch is in January, when BST is not in force. (However your EPOCH would also be completely off if you weren't in the UK.)

You should replace both your uses of datetime.fromtimestamp with datetime.utcfromtimestamp.

It's sad that datetime continues the awful time tradition of keeping times in local time. Calling them ‘naïve’ and taking away the DST flag just makes them even worse. Personally I can't stand to use datetime, preferring integer UTC timestamps for everything (converting to local timezones for formatting only).

Answer 2

Judging by your profile, you're in the UK. That means you're currently running on UTC+1 due to DST.

If I take your timestamp and run it through datetime.fromtimestamp on Python 2.6 (I know you use Python 3, but this is what I have), that shows me that it believes it refers to 2010-04-04 02:00:00 - and I'm in CEST, so that's UTC+2.

Running datetime.fromtimestamp(0), I get that the epoch is 1970-01-01 01:00:00. This then shows me that it is correctly adding only a single hour (since January 1st is outside of DST, and the epoch is midnight UTC on that date, it would be 01:00 here).

In other words, your problem is that you're sending in a time which has DST applied, but datetime_to_timestamp treats it as if DST didn't exist. timestamp_to_datetime, however, applies the DST.

Unfortunately, I don't know enough Python to know how you would solve this, but this should at least give you something to go on.