How to append on a dataframe with timezone aware timestamp column?

Question

I have a data frame with a timestamp column and a numeric column. I am able to append a new row to it if the timestamp column is timezone naive.

df = pd.DataFrame([[1,2],[3,4]], columns=['timestamp', 'number'])
df['timestamp']=pd.to_datetime(df['timestamp'])
df
#                       timestamp  number
# 0 1970-01-01 00:00:00.000000001       2
# 1 1970-01-01 00:00:00.000000003       4

df.append(df.loc[0])
#                       timestamp  number
# 0 1970-01-01 00:00:00.000000001       2
# 1 1970-01-01 00:00:00.000000003       4
# 0 1970-01-01 00:00:00.000000001       2

But if I set timezone for the timestamp column, and then try to append new rows, I get error.

df['timestamp']=df['timestamp'].apply(lambda x: x.tz_localize('utc'))
df
#                             timestamp  number
# 0 1970-01-01 00:00:00.000000001+00:00       2
# 1 1970-01-01 00:00:00.000000003+00:00       4
df.append(df.loc[0])
# Traceback (most recent call last):
#   File "<stdin>", line 1, in <module>
#   File "/Library/Python/2.7/site-packages/pandas-0.17.1-py2.7-macosx-10.10-intel.egg/pandas/core/frame.py", line 4231, in append
#     verify_integrity=verify_integrity)
#   File "/Library/Python/2.7/site-packages/pandas-0.17.1-py2.7-macosx-10.10-intel.egg/pandas/tools/merge.py", line 813, in concat
#     return op.get_result()
#   File "/Library/Python/2.7/site-packages/pandas-0.17.1-py2.7-macosx-10.10-intel.egg/pandas/tools/merge.py", line 995, in get_result
#     mgrs_indexers, self.new_axes, concat_axis=self.axis, copy=self.copy)
#   File "/Library/Python/2.7/site-packages/pandas-0.17.1-py2.7-macosx-10.10-intel.egg/pandas/core/internals.py", line 4456, in concatenate_block_managers
#     for placement, join_units in concat_plan]
#   File "/Library/Python/2.7/site-packages/pandas-0.17.1-py2.7-macosx-10.10-intel.egg/pandas/core/internals.py", line 4561, in concatenate_join_units
#     concat_values = com._concat_compat(to_concat, axis=concat_axis)
#   File "/Library/Python/2.7/site-packages/pandas-0.17.1-py2.7-macosx-10.10-intel.egg/pandas/core/common.py", line 2548, in _concat_compat
#     return _concat_compat(to_concat, axis=axis)
#   File "/Library/Python/2.7/site-packages/pandas-0.17.1-py2.7-macosx-10.10-intel.egg/pandas/tseries/common.py", line 256, in _concat_compat
#     return DatetimeIndex(np.concatenate([ x.tz_localize(None).asi8 for x in to_concat ]), tz=list(tzs)[0])
# AttributeError: 'numpy.ndarray' object has no attribute 'tz_localize'

Any help on how can I append new rows to a dataframe having timezone aware timespamp column will be greatly appreciated.

Answer 1

It's a bug in this pandas version (credit to this answer). As they state there, your solution can be:

df = df.astype(str).append(df.loc[0].astype(str))
df['timestamp'] = pd.to_datetime(df['timestamp'], utc=True)