Write and read Datetime to binary format in Python

Question

I want to store a list of datetimes in a binary file in Python.

EDIT: by "binary" I mean the best digital representation for each datatype. The application for this is to save GPS trackpoints composed by (unix-timestamp, latitude, longitude, elevation), so the whole structure is little-endian "Long, float, float, float", with four bytes to each value.

NOTE: I don't use "unix-timestamp" due to any affection to the Unix platform, but only as an unequivocal way to represent the value of a datetime.

Currently, I am doing like the code below, but besides some timezone confusion that I'm still working out (my timezone is -3), I believe converting to int and back might not be the right way, since datetime and datetime64 are native types in python/numpy, if I'm not mistaken. Thus, a datetime64 would need eight bytes instead of the four I am using for the (long)unix-timestamp.

import datetime
import calendar
import struct

now = datetime.datetime.now()
print now

stamp = calendar.timegm(now.utctimetuple())
print stamp

binarydatetime = struct.pack('<L', stamp)
recoverstamp = struct.unpack('<L', binarydatetime)[0]
print recoverstamp

recovernow = datetime.datetime.fromtimestamp(recoverstamp)
print recovernow

So the main question is: "is this the pythonic way to converting naive datetime to binary and back?"

And the aditional question is: "if everything in this code is supposed to be naive, why do I have a timezone offset?"

Thanks for reading!

Answer 1

I have found a way using the Unix timestamp and storing it as an integer. This works for me because I don't need a subsecond resolution, but I think long integers would allow for microsecond resolution with some modifications of the code.

The changes from my original consist in replacing calendar.timegm by time.mktime and also utctimetuple by timetuple, to keep everything naive.

This:

import datetime
import struct
import time

now = datetime.datetime.now()
print now

stamp = time.mktime(now.timetuple())
print stamp

recoverstamp = datetime.datetime.fromtimestamp(stamp)
print recoverstamp

binarydatetime = struct.pack('<L', stamp)
recoverbinstamp = struct.unpack('<L', binarydatetime)[0]
print recoverbinstamp

recovernow = datetime.datetime.fromtimestamp(recoverbinstamp)
print recovernow

Returns this:

2013-09-02 11:06:28.064000
1378130788.0
2013-09-02 11:06:28
1378130788
2013-09-02 11:06:28

From this, I can easily write the packed binarydatetime to file, and read it back later.

Answer 2

By far the most simple solution is to use Temporenc: http://temporenc.readthedocs.org/

It takes care of all the encoding/decoding and allows you to write a Python datetime object to a file:

now = datetime.datetime.now()
temporenc.pack(fp, now)

To read it back, this suffices:

dt = temporenc.unpack(fp)
print(dt)