Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is SciPy acting very differently in IPython and Python?

I wrote this test script:

import numpy as np
import scipy.linalg

n = 130
r = np.array(np.random.normal(size=(n, n)), dtype=np.float32)
e = scipy.linalg.eig(r, left=False, right=False)
print e.mean()

Running it using IPython, the code always succeeds in a fraction of a second (I tried it about a dozen times)

With Python, the code always fails to converge (or just hangs, for some larger n) with a message like

Traceback (most recent call last):
  File "strange.py", line 6, in <module>
    e = scipy.linalg.eig(r, left=False, right=False)
  File "/usr/lib/python2.7/dist-packages/scipy/linalg/decomp.py", line 179, in eig
    "with order >= %d have converged)" % info)
numpy.linalg.linalg.LinAlgError: eig algorithm did not converge (only eigenvalues with order >= 130 have converged)

What explains this difference in the behavior of Python and IPython? The relevant software versions are:

  • Ubuntu 12.04, 64-bit
  • Numpy 1.6.1
  • SciPy 0.9.0
  • Python 2.7.3
  • IPython 0.12.1

Edit

I observed this behavior only with single precision and n >= 130. If n = 129, the code works in both Python and IPython.

Adding np.random.seed(1234) after the imports gives the same result: IPython converges, while Python does not.

scipy.linalg.__file__ = '/usr/lib/python2.7/dist-packages/scipy/linalg/__init__.pyc' in both. Despite this, I would guess that IPython and Python somehow manage to pull in different LAPACK versions, but how?

The way I noticed this oddity is that I was experimenting in IPython, and then pasting the code into a *.py file that I run with Python. You can imagine how confused I was for a while.

Edit 2.

np.geterr() is {'divide': 'warn', 'invalid': 'warn', 'over': 'warn', 'under': 'ignore'} in both Python and IPython

$ ls -l /etc/alternatives/libblas.so
lrwxrwxrwx 1 root root 37 Jun 29 18:21 /etc/alternatives/libblas.so -> /usr/lib/openblas-base/libopenblas.so
like image 694
MWB Avatar asked Dec 10 '13 12:12

MWB


1 Answers

It could be that the LD_LIBRARY_PATH is different when you are using IPython. This could result in using different libraries. You can check this in both Python and IPython and see if they are identical:

import os
print os.environ['LD_LIBRARY_PATH']
like image 153
SKV Avatar answered Oct 29 '22 06:10

SKV