Matplotlib: How to skip a range of hours when plotting with a datetime axis?

Question

I have tick-by-tick data of a financial instrument, which I am trying to plot using matplotlib. I am working with pandas and the data is indexed with DatetimeIndex.

The problem is, when I try to plot multiple trading days I can't skip the range of time between the market closing time and next day's opening (see the example), which of course I am not interested in.

Is there a way to make matplotlib ignore this and just "stick" together the closing quote with the following day's opening? I tried to pass a custom range of time:

plt.xticks(time_range)

But the result is the same. Any ideas how to do this?

# Example data
instrument = pd.DataFrame(data={
    'Datetime': [
        dt.datetime.strptime('2018-01-11 11:00:11', '%Y-%m-%d %H:%M:%S'),
        dt.datetime.strptime('2018-01-11 13:02:17', '%Y-%m-%d %H:%M:%S'),
        dt.datetime.strptime('2018-01-11 16:59:14', '%Y-%m-%d %H:%M:%S'),

        dt.datetime.strptime('2018-01-12 11:00:11', '%Y-%m-%d %H:%M:%S'),
        dt.datetime.strptime('2018-01-12 13:15:24', '%Y-%m-%d %H:%M:%S'),
        dt.datetime.strptime('2018-01-12 16:58:43', '%Y-%m-%d %H:%M:%S')
    ],
    'Price': [127.6, 128.1, 127.95, 129.85, 129.7, 131.2],
    'Volume': [725, 146, 48, 650, 75, 160]
}).set_index('Datetime')

plt.figure(figsize=(10,5))
top = plt.subplot2grid((4,4), (0, 0), rowspan=3, colspan=4)
bottom = plt.subplot2grid((4,4), (3,0), rowspan=1, colspan=4)
top.plot(instrument.index, instrument['Price'])
bottom.bar(instrument.index, instrument['Volume'], 0.005) 

top.xaxis.get_major_ticks()
top.axes.get_xaxis().set_visible(False)
top.set_title('Example')
top.set_ylabel('Price')
bottom.set_ylabel('Volume')

Answer 1

TL;DR

Replace the matplotlib plotting functions:

top.plot(instrument.index, instrument['Price'])
bottom.bar(instrument.index, instrument['Volume'], 0.005)

With these ones:

top.plot(range(instrument.index.size), instrument['Price'])
bottom.bar(range(instrument.index.size), instrument['Volume'], width=1)

Or with these pandas plotting functions (only the x-axis limits will look different):

instrument['Price'].plot(use_index=False, ax=top)
instrument['Volume'].plot.bar(width=1, ax=bottom)

Align both plots by sharing the x-axis with sharex=True and set up the ticks as you would like them using the dataframe index, as shown in the example further below.

---

Let me first create a sample dataset and show what it looks like if I plot it using matplotlib plotting functions like in your example where the DatetimeIndex is used as the x variable.

Create sample dataset

The sample data is created using the pandas_market_calendars package to create a realistic DatetimeIndex with a minute-by-minute frequency that spans several weekdays and a weekend.

import numpy as np                        # v 1.19.2
import pandas as pd                       # v 1.1.3
import matplotlib.pyplot as plt           # v 3.3.2
import matplotlib.ticker as ticker
import pandas_market_calendars as mcal    # v 1.6.1

# Create datetime index with a 'minute start' frequency based on the New
# York Stock Exchange trading hours (end date is inclusive)
nyse = mcal.get_calendar('NYSE')
nyse_schedule = nyse.schedule(start_date='2021-01-07', end_date='2021-01-11')
nyse_dti = mcal.date_range(nyse_schedule, frequency='1min', closed='left')\
               .tz_convert(nyse.tz.zone)
# Remove timestamps of closing times to create a 'period start' datetime index
nyse_dti = nyse_dti.delete(nyse_dti.indexer_at_time('16:00'))

# Create sample of random data consisting of opening price and
# volume of financial instrument traded for each period
rng = np.random.default_rng(seed=1234)  # random number generator
price_change = rng.normal(scale=0.1, size=nyse_dti.size)
price_open = 127.5 + np.cumsum(price_change)
volume = rng.integers(100, 10000, size=nyse_dti.size)
df = pd.DataFrame(data=dict(Price=price_open, Volume=volume), index=nyse_dti)

df.head()

#                             Price       Volume
#  2021-01-07 09:30:00-05:00  127.339616  7476
#  2021-01-07 09:31:00-05:00  127.346026  3633
#  2021-01-07 09:32:00-05:00  127.420115  1339
#  2021-01-07 09:33:00-05:00  127.435377  3750
#  2021-01-07 09:34:00-05:00  127.521752  7354

Plot data with matplotlib using the DatetimeIndex

This sample data can now be plotted using matplotlib plotting functions like in your example, but note that the subplots are created by using plt.subplots with the sharex=True argument. This aligns the line with the bars correctly and makes it possible to use the interactive interface of matplotlib with both subplots.

# Create figure and plots using matplotlib functions
fig, (top, bot) = plt.subplots(2, 1, sharex=True, figsize=(10,5),
                               gridspec_kw=dict(height_ratios=[0.75,0.25]))
top.plot(df.index, df['Price'])
bot.bar(df.index, df['Volume'], 0.0008)

# Set title and labels
top.set_title('Matplotlib plots with unwanted gaps', pad=20, size=14, weight='semibold')
top.set_ylabel('Price', labelpad=10)
bot.set_ylabel('Volume', labelpad=10);

Plot data with matplotlib without any gaps by using a range of integers

The problem of these gaps can be solved by simply ignoring the DatetimeIndex and using a range of integers instead. Most of the work then lies in creating appropriate tick labels. Here is an example:

# Create figure and matplotlib plots with some additional formatting
fig, (top, bot) = plt.subplots(2, 1, sharex=True, figsize=(10,5),
                               gridspec_kw=dict(height_ratios=[0.75,0.25]))
top.plot(range(df.index.size), df['Price'])
top.set_title('Matplotlib plots without any gaps', pad=20, size=14, weight='semibold')
top.set_ylabel('Price', labelpad=10)
top.grid(axis='x', alpha=0.3)
bot.bar(range(df.index.size), df['Volume'], width=1)
bot.set_ylabel('Volume', labelpad=10)

# Set fixed major and minor tick locations
ticks_date = df.index.indexer_at_time('09:30')
ticks_time = np.arange(df.index.size)[df.index.minute == 0][::2] # step in hours
bot.set_xticks(ticks_date)
bot.set_xticks(ticks_time, minor=True)

# Format major and minor tick labels
labels_date = [maj_tick.strftime('\n%d-%b').replace('\n0', '\n')
               for maj_tick in df.index[ticks_date]]
labels_time = [min_tick.strftime('%I %p').lstrip('0').lower()
               for min_tick in df.index[ticks_time]]
bot.set_xticklabels(labels_date)
bot.set_xticklabels(labels_time, minor=True)
bot.figure.autofmt_xdate(rotation=0, ha='center', which='both')

Create dynamic ticks for interactive plots

If you like to use the interactive interface of matplotlib (with pan/zoom), you will need to use locators and formatters from the matplotlib ticker module. Here is an example of how to set the ticks, where the major ticks are fixed and formatted like above but the minor ticks are generated automatically as you zoom in/out of the plot:

# Set fixed major tick locations and automatic minor tick locations
ticks_date = df.index.indexer_at_time('09:30')
bot.set_xticks(ticks_date)
bot.xaxis.set_minor_locator(ticker.AutoLocator())

# Format major tick labels
labels_date = [maj_tick.strftime('\n%d-%b').replace('\n0', '\n')
               for maj_tick in df.index[ticks_date]]
bot.set_xticklabels(labels_date)

# Format minor tick labels
def min_label(x, pos):
    if 0 <= x < df.index.size:
        return df.index[int(x)].strftime('%H:%M')
min_fmtr = ticker.FuncFormatter(min_label)
bot.xaxis.set_minor_formatter(min_fmtr)

bot.figure.autofmt_xdate(rotation=0, ha='center', which='both')

---

Documentation: example of an alternative solution; datetime string format codes

Answer 2

Maybe use https://pypi.org/project/mplfinance/
Allows mimicking the usual financial plots you see in most services.

When you call the mplfinance mpf.plot() function, there is a kwarg show_nontrading, which by default is set to False so that these unwanted gaps are automatically not plotted. (To plot them, set show_nontrading=True).