pandas scatter plotting datetime

Question

I have a dataframe with two columns of datetime.time's. I'd like to scatter plot them. I'd also like the axes to display the times, ideally. But

df.plot(kind='scatter', x='T1', y='T2')

dumps a bunch of internal plot errors ending with a KeyError on 'T1'.

Alternatively, I try

plt.plot_date(x=df.loc[:,'T1'], y=df.loc[:,'T2'])
plt.show()

and I get 'Exception in Tkinter callback' with a long stack crawl ending in

return _from_ordinalf(x, tz)
  File "/usr/lib/python3/dist-packages/matplotlib/dates.py", line 224, in _from_ordinalf
microsecond, tzinfo=UTC).astimezone(tz)
TypeError: tzinfo argument must be None or of a tzinfo subclass, not type 'str'

Any pointers?

Answer 1

Not an answer, but I can't edit the question or put this much in a comment, I think.

Here is a reproducible example:

from datetime import datetime
import pandas as pd
df = pd.DataFrame({'x': [datetime.now() for _ in range(10)], 'y': range(10)})
df.plot(x='x', y='y', kind='scatter')

This gives KeyError: 'x'.

Interestingly, you do get a plot with just df.plot(x='x', y='y'); it chooses poorly for the default x range because the times are just nanoseconds apart, which is weird, but that's a separate issue. It seems like if you can make a line graph, you should be able to make a scatterplot too.

There is a pandas github issue about this problem, but it was closed for some reason. I'm going to go comment there and see if we can re-start that conversation.

Is there some clever work-around for this? If so, what?

Answer 2

building on Mike N's answer...convert to unix time to scatter properly, then transform your axis labels back from int64s to strings:

type(df.ts1[0])

pandas.tslib.Timestamp

df['t1'] = df.ts1.astype(np.int64)
df['t2'] = df.ts2.astype(np.int64)

fig, ax = plt.subplots(figsize=(10,6))
df.plot(x='t1', y='t2', kind='scatter', ax=ax)
ax.set_xticklabels([datetime.fromtimestamp(ts / 1e9).strftime('%H:%M:%S') for ts in ax.get_xticks()])
ax.set_yticklabels([datetime.fromtimestamp(ts / 1e9).strftime('%H:%M:%S') for ts in ax.get_yticks()])
plt.show()