matplotlib bar3d clipping problems

Question

I am trying to create a 3D bar graph with Matplotlib 1.2.0 and Python 2.7.3. I followed the advice in http://www.mail-archive.com/[email protected]/msg19740.html and plotted the bar one by one, but I am still getting rendering problems (i.e., bars on top of each other).

Moreover, I get the following when I invoke my code:

/usr/apps/python/lib/python2.7/site-packages/mpl_toolkits/mplot3d/axes3d.py:1476: RuntimeWarning: divide by zero encountered in divide for n in normals])

/usr/apps/python/lib/python2.7/site-packages/mpl_toolkits/mplot3d/axes3d.py:1476: RuntimeWarning: invalid value encountered in divide for n in normals])

My questions:

1. Are these serious warnings? Do I need to look into them and try to eliminate them? How do I eliminate them?
2. What is the difference between zsort='max' and zsort='average'?
3. What else can I do to eliminate rendering problems?

Thanks in advance!

Here is my code:

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.colors as colors
import matplotlib.cm as cmx

# my data
dat = [2.31778665482167e-310, 0.006232785101850947, 0.0285075971030949, 0.0010248181570355695, 0.0048776795767614825, 0.02877090365176044, 0.002459331469834533, 0.0008594610645495889, 0.002919824084878003, 0.000968081117692596, 0.0, 0.0, 0.0319623949119874, 0.00568752311279771, 0.009994801469036968, 0.03248018520506219, 0.006686905726805326, 0.005987863156039365, 0.0072955095915350045, 0.005568911905473998, 0.0, 0.0, 0.0, 0.028483143996551524, 0.031030793902192794, 0.06125216053962635, 0.02935971973938871, 0.028507530280092265, 0.030112963748812088, 0.028293406731749605, 0.0, 0.0, 0.0, 0.0, 0.004510645022825792, 0.028998119822468988, 0.0013993630391143715, 0.0010726572949244424, 0.002288215944285159, 0.0006513973584945584, 0.0, 1.1625e-320, 1.15348834e-316, 2.3177866547513e-310, 0.0, 0.03148966953869102, 0.005215047563268979, 0.004491716298086729, 0.006010166308872446, 0.005186976949223524, 0.0, 0.0, 0.0, 0.0, 0.0, 1.107e-320, 0.02983657915729719, 0.028893006725328373, 0.030526067389954753, 0.028629390713739978, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0015217840289869456, 0.002751587509779179, 0.001413669523724954, 1.15348834e-316, 2.3177866547513e-310, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0024680339073824705, 0.0008254364860386303, 0.0, 0.0, 0.0, 9.965e-321, 1.15348834e-316, 2.3177866547513e-310, 0.0, 0.0, 0.0, 0.002621588539481613, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9.41e-321, 1.15348834e-316, 2.3177866547513e-310]
dat = np.reshape(dat,[10,10],order='F')

lx = len(dat[0])
ly = len(dat[:,0])
n = lx*ly

# generate colors
cm = plt.get_cmap('jet')
vv = range(len(dat))
cNorm = colors.Normalize(vmin=0, vmax=vv[-1])
scalarMap = cmx.ScalarMappable(norm=cNorm, cmap=cm)
colorVals = [scalarMap.to_rgba(i) for i in range(ly)]

# generate plot data
xpos = np.arange(0,lx,1)  
ypos = np.arange(0,ly,1)
xpos, ypos = np.meshgrid(xpos+0.25, ypos+0.25)
xpos = xpos.flatten()
ypos = ypos.flatten()
zpos = np.zeros(n)
dx = 0.5*np.ones_like(zpos)
dy = dx.copy()
dz = dat.flatten()
cc = np.tile(range(lx), (ly,1))
cc = cc.T.flatten()

# generate plot
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
opacity = 1

for i in range(n):
    ax.bar3d(xpos[i], ypos[i], zpos[i], dx[i], dy[i], dz[i],
             color=colorVals[cc[i]], alpha=opacity, zsort='max')

plt.autoscale(enable=True, axis='both', tight=True)
plt.grid()
plt.show(block=False)

Answer 1

I think marisano 's answer has a problem of rendering with various height because it uses the Euclidean distance from the top of a bar to the camera position and substract this value from maximal z_order, I don't think it is the correct way. Finally I adopt the same measurement of z_order from apodemus and updated it below:

z_order = np.multiply([xpos,ypos, np.zeros_like(xpos)],camera).sum(0)

and

pl._sort_zpos = z_order[i]

Now it works in my case.

Answer 2

This answer is a quick fix solution that allows you to produce certain types of 3D bar charts in matplotlib with correct rendering. The trick is to A) plot the bars individually, B) hack the zsort algorithm to force sorting the bars w.r.t. the "distance" from the camera. This can be done by overwrite the _sort_zpos attribute of the PolyCollection3D instance returned by ax.bar3d. The following code demonstrates the solution using data drawn from a 2D Gaussian.

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from scipy.stats import multivariate_normal


def sph2cart(r, theta, phi):
    '''spherical to cartesian transformation.'''
    x = r * np.sin(theta) * np.cos(phi)
    y = r * np.sin(theta) * np.sin(phi)
    z = r * np.cos(theta)
    return x, y, z

def sphview(ax):
    '''returns the camera position for 3D axes in spherical coordinates'''
    r = np.square(np.max([ax.get_xlim(), ax.get_ylim()], 1)).sum()
    theta, phi = np.radians((90-ax.elev, ax.azim))
    return r, theta, phi

def ravzip(*itr):
    '''flatten and zip arrays'''
    return zip(*map(np.ravel, itr))

#Generate data
res = 15
sl = slice(-3, 3, complex(res))
Y, X = np.mgrid[sl, sl]
grid = np.array([X, Y])
(dx,), (dy,) = 0.8*np.diff(X[0,:2]), 0.8*np.diff(Y[:2,0])

#2D Gaussian
mu = (0, 0)
covm = np.array([[ 0.8,  0.3],
                 [ 0.3,  0.5]])
rv = multivariate_normal(mu, covm)
Zg = rv.pdf(grid.transpose(1,2,0)).T

#generate the figure
fig, (ax1, ax2) = plt.subplots(1,2, subplot_kw=dict(projection='3d'))

#standard bar3d
ax1.set_title('Standard')
ax1.bar3d(X.ravel(), Y.ravel(), np.zeros(X.size), dx, dy, Zg.ravel(), '0.85')

#Fixed bar3d
ax2.set_title('Fixed')

xyz = np.array(sph2cart(*sphview(ax2)), ndmin=3).T       #camera position in xyz
zo = np.multiply([X, Y, np.zeros_like(Zg)], xyz).sum(0)  #"distance" of bars from camera

bars = np.empty(X.shape, dtype=object)
for i, (x,y,dz,o) in enumerate(ravzip(X, Y, Zg, zo)):
    j, k = divmod(i, res)
    bars[j, k] = pl = ax2.bar3d(x, y, 0, dx, dy, dz, '0.85')
    pl._sort_zpos = o

plt.show()

Which produces the following figure:

Note: This will only work for the initial viewing angle. If you rotate the axes, you will have to set the _sort_zpos for all the bars again and redraw the canvas to fix the rendering.

Answer 3

I took apodemus's code, which apparently works, unpacked it, and applied it to the original question to provide a direct answer for it. My code could certainly be cleaned up - particularly the loop in getDistances() - but it does solve the presented problem, and should be much easier to follow. To surmise, the distance to the viewer, i.e., the camera distance, has to be determined by calling sphview() and sph2cart(). Then the distances of all of the bars from the camera must be calculated by calling getdistances(). Thereafter, the bars should be drawn one and a time, and crucially, each bar's z-order must be explicitly set based on the previously determined distances.

If the resultant graph is rotated live in the plot window, it may not be updated correctly. However, presetting the location of the camera allows arbitrary initial views to be plotted without error. (There may well be a callback mechanism that could be invoked to cause the z-orders of the bars to be explicitly recalculated, but I have no knowledge of such an API.) The location of the camera can be preset by passing azim and elev to fig.add_subplot(). Its distance can be changed by setting the dist field of the Axes instance returned by fig.add_subplot().

Following is the graph produced by the updated code applied to the original question:

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.colors as colors
import matplotlib.cm as cmx

# from apodemus's Stackoverflow answer,
# https://stackoverflow.com/questions/18602660/matplotlib-bar3d-clipping-problems
def sph2cart(r, theta, phi):
    '''spherical to Cartesian transformation.'''
    x = r * np.sin(theta) * np.cos(phi)
    y = r * np.sin(theta) * np.sin(phi)
    z = r * np.cos(theta)
    return x, y, z

def sphview(ax):
    '''returns the camera position for 3D axes in spherical coordinates'''
    r = np.square(np.max([ax.get_xlim(), ax.get_ylim()], 1)).sum()
    theta, phi = np.radians((90-ax.elev, ax.azim))
    return r, theta, phi
#
# end of apodemus's code

def getDistances(view):
    distances  = []
    a = np.array((xpos, ypos, dz))
    for i in range(len(xpos)):
        distance = (a[0, i] - view[0])**2 + (a[1, i] - view[1])**2 + (a[2, i] - view[2])**2
        distances.append(np.sqrt(distance))
    return distances

# ================================================================

# my data
dat = [2.31778665482167e-310, 0.006232785101850947, 0.0285075971030949, 0.0010248181570355695, 0.0048776795767614825, 0.02877090365176044, 0.002459331469834533, 0.0008594610645495889, 0.002919824084878003, 0.000968081117692596, 0.0, 0.0, 0.0319623949119874, 0.00568752311279771, 0.009994801469036968, 0.03248018520506219, 0.006686905726805326, 0.005987863156039365, 0.0072955095915350045, 0.005568911905473998, 0.0, 0.0, 0.0, 0.028483143996551524, 0.031030793902192794, 0.06125216053962635, 0.02935971973938871, 0.028507530280092265, 0.030112963748812088, 0.028293406731749605, 0.0, 0.0, 0.0, 0.0, 0.004510645022825792, 0.028998119822468988, 0.0013993630391143715, 0.0010726572949244424, 0.002288215944285159, 0.0006513973584945584, 0.0, 1.1625e-320, 1.15348834e-316, 2.3177866547513e-310, 0.0, 0.03148966953869102, 0.005215047563268979, 0.004491716298086729, 0.006010166308872446, 0.005186976949223524, 0.0, 0.0, 0.0, 0.0, 0.0, 1.107e-320, 0.02983657915729719, 0.028893006725328373, 0.030526067389954753, 0.028629390713739978, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0015217840289869456, 0.002751587509779179, 0.001413669523724954, 1.15348834e-316, 2.3177866547513e-310, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0024680339073824705, 0.0008254364860386303, 0.0, 0.0, 0.0, 9.965e-321, 1.15348834e-316, 2.3177866547513e-310, 0.0, 0.0, 0.0, 0.002621588539481613, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 9.41e-321, 1.15348834e-316, 2.3177866547513e-310]
dat = np.reshape(dat,[10,10],order='F')

lx = len(dat[0])
ly = len(dat[:,0])
n = lx*ly

# generate colors
cm = plt.get_cmap('jet')
vv = range(len(dat))
cNorm = colors.Normalize(vmin=0, vmax=vv[-1])
scalarMap = cmx.ScalarMappable(norm=cNorm, cmap=cm)
colorVals = [scalarMap.to_rgba(i) for i in range(ly)]

# generate plot data
xpos = np.arange(0,lx,1)  
ypos = np.arange(0,ly,1)
xpos, ypos = np.meshgrid(xpos+0.25, ypos+0.25)
xpos = xpos.flatten()
ypos = ypos.flatten()
zpos = np.zeros(n)
dx = 0.5*np.ones_like(zpos)
dy = dx.copy()
dz = dat.flatten()
cc = np.tile(range(lx), (ly,1))
cc = cc.T.flatten()

# generate plot
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
opacity = 1

# Get the camera's location in Cartesian coordinates.
x1, y1, z1 = sph2cart(*sphview(ax))
camera = np.array((x1,y1,0))
# Calculate the distance of each bar from the camera.
z_order = getDistances(camera)
max = max(z_order)

for i in range(n):
    pl = ax.bar3d(xpos[i], ypos[i], zpos[i], dx[i], dy[i], dz[i],
             color=colorVals[cc[i]], alpha=opacity, zsort='max')
    # The z-order must be set explicitly.
    #
    # z-order values are somewhat backwards in magnitude, in that the largest
    # value is closest to the camera - unlike, in say, a coordinate system.
    # Therefore, subtracting the maximum distance from the calculated distance
    # inverts the z-order to the proper form.
    pl._sort_zpos = max - z_order[i]

plt.autoscale(enable=True, axis='both', tight=True)
plt.grid()
plt.show()

This approach (as opposed to using Mayavi to handle 3D drawing, for example) allows the matplotlib appearance to be retained in the graph itself, as well as its adornments such as axes numbers, labels, and legends.

Answer 4

This isn't the answer that you are looking for, but I think that this might be a bug in matplotlib. I think that the same problem was encountered here. The problem was described as "intractable" according to the mplot3d FAQ.

But to me it doesn't seem intractable. You simple need to figure out which object is closer to the viewer and set the z-order accordingly. So, I think that the problem might just be a bug.

If I take the matplotlib 3D histogram example and just change "bins=4" to "bins=6" or a higher number, then I get the same "axes3d.py:1476: RuntimeWarning: invalid value encountered in divide / for n in normals])". Also, I can reproduce the wrong z-order of the bars (check out the tall guy near the front who jumps in front of his short friend):

The incorrect ordering of the bars seems linked to the divide by zero error, since the plots look just fine when I use a smaller number of bins.

Line 1476 in axes.py is:

shade = np.array([np.dot(n / proj3d.mod(n), [-1, -1, 0.5]) for n in normals])

Basically, I think it is trying to figure out the shading using the normal vectors to each face. But, one or more of the normal vectors is zero, which should not be the case. So, I think that this is just some bug in matplotlib that can probably be fixed by someone with more programming skills than myself.

The mplot3d FAQ is correct that MayaVI can be used if you want a better 3D engine. I used

from mayavi import mlab
mlab.barchart(xpos,ypos,dz*100)

to generate a plot of your data:

I hope that this gets figured out soon. I would like to make some similar 3D barcharts in the near future.