Statsmodels mosaic plot ValueError: cannot convert float NaN to integer

Question

I have a simple pandas DataFrame, for which I would like to create a mosaic plot. Here is my code:

import pandas as pd
from statsmodels.graphics.mosaicplot import mosaic 

mydata = pd.DataFrame({'id2': {64: 'Angelica', 
                               65: 'DXW_UID', 66: 'casuid01', 
                               67: 'casuid01', 68: 'EC93_uid', 
                               69: 'EC93_uid', 70: 'EC93_uid', 
                               60: 'DXW_UID',  61: 'AtmosFox', 
                               62: 'DXW_UID', 63: 'DXW_UID'}, 
                       'id1': {64: 'TGP', 
                               65: 'Retention01', 66: 'default',
                               67: 'default', 68: 'Musa_EC_9_3', 
                               69: 'Musa_EC_9_3', 70: 'Musa_EC_9_3', 
                               60: 'default', 61: 'default', 
                               62: 'default', 63: 'default'}})

mydata
            id1       id2
60      default   DXW_UID
61      default  AtmosFox
62      default   DXW_UID
63      default   DXW_UID
64          TGP  Angelica
65  Retention01   DXW_UID
66      default  casuid01
67      default  casuid01
68  Musa_EC_9_3  EC93_uid
69  Musa_EC_9_3  EC93_uid
70  Musa_EC_9_3  EC93_uid

[11 rows x 2 columns]

I can create a mosaic plot just fine when I exclude row 64.

mosaic(mydata[mydata.id1!='TGP'], ['id1','id2'])
(<matplotlib.figure.Figure object at 0x11E0D3B0>, OrderedDict([(('default', 'DXW_UID'), (0.0, 0.0, 0.594059405940594, 0.49504950495049505)), (('default', 'AtmosFox'), (0.0, 0.49834983498349833, 0.594059405940594, 0.16501650165016499)), (('default', 'casuid01'), (0.0, 0.66666666666666663, 0.594059405940594, 0.33003300330033009)), (('default', 'EC93_uid'), (0.0, 1.0, 0.594059405940594, 0.0)), (('Retention01', 'DXW_UID'), (0.599009900990099, 0.0, 0.09900990099009899, 0.99009900990099009)), (('Retention01', 'AtmosFox'), (0.599009900990099, 0.99339933993399343, 0.09900990099009899, 0.0)), (('Retention01', 'casuid01'), (0.599009900990099, 0.99669966996699666, 0.09900990099009899, 0.0)), (('Retention01', 'EC93_uid'), (0.599009900990099, 1.0, 0.09900990099009899, 0.0)), (('Musa_EC_9_3', 'DXW_UID'), (0.7029702970297029, 0.0, 0.29702970297029707, 0.0)), (('Musa_EC_9_3', 'AtmosFox'), (0.7029702970297029, 0.0033003300330033004, 0.29702970297029707, 0.0)), (('Musa_EC_9_3', 'casuid01'), (0.7029702970297029, 0.0066006600660066007, 0.29702970297029707, 0.0)), (('Musa_EC_9_3', 'EC93_uid'), (0.7029702970297029, 0.0099009900990099011, 0.29702970297029707, 0.99009900990099009))]))

The plot comes out fine (with the exception of some of the labels looking a little funny--but that's not the issue).

The errors occur when I include row 64. My questions are, why does this row cause this error, and how can I fix it? I can see that the error occurs when trying to draw the image, but it is not at all obvious where the NaN is coming from, especially since the plot before worked just fine.

mosaic(mydata, ['id1','id2'])
(<matplotlib.figure.Figure object at 0x11D13ED0>, OrderedDict([(('default', 'DXW_UID'), (0.0, 0.0, 0.5373936408419167, 0.49342105263157893)), (('default', 'AtmosFox'), (0.0, 0.49671052631578938, 0.5373936408419167, 0.16447368421052627)), (('default', 'casuid01'), (0.0, 0.66447368421052622, 0.5373936408419167, 0.32894736842105265)), (('default', 'Angelica'), (0.0, 0.99671052631578938, 0.5373936408419167, 0.0)), (('default', 'EC93_uid'), (0.0, 1.0, 0.5373936408419167, 0.0)), (('TGP', 'DXW_UID'), (0.5423197492163009, 0.0, 0.08956560680698614, 0.0)), (('TGP', 'AtmosFox'), (0.5423197492163009, 0.0032894736842105261, 0.08956560680698614, 0.0)), (('TGP', 'casuid01'), (0.5423197492163009, 0.0065789473684210523, 0.08956560680698614, 0.0)), (('TGP', 'Angelica'), (0.5423197492163009, 0.0098684210526315784, 0.08956560680698614, 0.98684210526315785)), (('TGP', 'EC93_uid'), (0.5423197492163009, 1.0, 0.08956560680698614, 0.0)), (('Retention01', 'DXW_UID'), (0.6368114643976712, 0.0, 0.08956560680698614, 0.98684210526315785)), (('Retention01', 'AtmosFox'), (0.6368114643976712, 0.99013157894736836, 0.08956560680698614, 0.0)), (('Retention01', 'casuid01'), (0.6368114643976712, 0.99342105263157876, 0.08956560680698614, 0.0)), (('Retention01', 'Angelica'), (0.6368114643976712, 0.99671052631578938, 0.08956560680698614, 0.0)), (('Retention01', 'EC93_uid'), (0.6368114643976712, 1.0, 0.08956560680698614, 0.0)), (('Musa_EC_9_3', 'DXW_UID'), (0.7313031795790416, 0.0, 0.2686968204209583, 0.0)), (('Musa_EC_9_3', 'AtmosFox'), (0.7313031795790416, 0.0032894736842105261, 0.2686968204209583, 0.0)), (('Musa_EC_9_3', 'casuid01'), (0.7313031795790416, 0.0065789473684210523, 0.2686968204209583, 0.0)), (('Musa_EC_9_3', 'Angelica'), (0.7313031795790416, 0.0098684210526315784, 0.2686968204209583, 0.0)), (('Musa_EC_9_3', 'EC93_uid'), (0.7313031795790416, 0.013157894736842105, 0.2686968204209583, 0.98684210526315785))]))

When I run the above, I get this Traceback:

  File "C:\Python27\lib\site-packages\matplotlib\backends\backend_qt4.py", line 374, in idle_draw
    self.draw()
  File "C:\Python27\lib\site-packages\matplotlib\backends\backend_qt4agg.py", line 154, in draw
    FigureCanvasAgg.draw(self)
  File "C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py", line 451, in draw
    self.figure.draw(self.renderer)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\figure.py", line 1034, in draw
    func(*args)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\axes.py", line 2086, in draw
    a.draw(renderer)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\axis.py", line 1096, in draw
    tick.draw(renderer)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\axis.py", line 241, in draw
    self.label1.draw(renderer)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\text.py", line 598, in draw
    ismath=ismath, mtext=self)
  File "C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py", line 188, in draw_text
    font.get_image(), np.round(x - xd), np.round(y + yd) + 1, angle, gc)
ValueError: cannot convert float NaN to integer
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\matplotlib\backends\backend_qt4.py", line 299, in resizeEvent
    self.draw()
  File "C:\Python27\lib\site-packages\matplotlib\backends\backend_qt4agg.py", line 154, in draw
    FigureCanvasAgg.draw(self)
  File "C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py", line 451, in draw
    self.figure.draw(self.renderer)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\figure.py", line 1034, in draw
    func(*args)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\axes.py", line 2086, in draw
    a.draw(renderer)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\axis.py", line 1096, in draw
    tick.draw(renderer)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\axis.py", line 241, in draw
    self.label1.draw(renderer)
  File "C:\Python27\lib\site-packages\matplotlib\artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "C:\Python27\lib\site-packages\matplotlib\text.py", line 598, in draw
    ismath=ismath, mtext=self)
  File "C:\Python27\lib\site-packages\matplotlib\backends\backend_agg.py", line 188, in draw_text
    font.get_image(), np.round(x - xd), np.round(y + yd) + 1, angle, gc)
ValueError: cannot convert float NaN to integer

I ran the above code in the spyder IDE, with default settings.

A similar issue was addressed here, and numerical underflow was the culprit. However, if that is the case here, it is not at all obvious why.

Answer 1

According to the docs the first parameter should be a contingency table. The fact that your way of doing things works at all seems to be an undocumented feature.

The behaviour you're seeing (including your "funny" looking labels) is because many of the entries in your contingency table are zero, and something in the labelling code of mosiac is having a hard time with that.

To see this, convert your DataFrame to a contingency table:

In [161]: pd.crosstab(mydata.id1, mydata.id2)
Out[161]: 
id2          Angelica  AtmosFox  DXW-UID  EC93-uid  casuid01
id1                                                         
Musa-EC-9-3         0         0        0         3         0
Retention01         0         0        1         0         0
TGP                 1         0        0         0         0
default             0         1        3         0         2

And add a "little bit" to all those zeros. The mosiac then works fine.

In [165]: ct = pd.crosstab(mydata.id1, mydata.id2)
In [166]: ctplus = ct + 1
In [167]: mosaic(ctplus.unstack())

Which results in the rather beautiful:

The tiny downside is that it's wrong! But you can remedy that by doing

ctplus = ct + 1e-8

to just add a tiny bit to all those zeros. The plot still works (but looks ugly because the labels on all those zero tiles of the mosaic are all on top of each other):