LookupError: unknown encoding: idna - only from subprocess.call

Question

Im trying to invoke a python 2.7 script from an Iron python one using subprocess.call. the invoked process uses imports that are not supported in IronPython like scikit-learn and requests.

Currently im trying to invoke the python 2.7 script using this comand:

sub_ret_val = subprocess.call("C:/Documents and Settings/avishay/workspace/BursaProject_V1_31/brainManager_toFile.py", shell = True, stderr = subprocess.STDOUT)

but im getting:

LookupError: unknown encoding: idna

now, this script works fine if I run it from the windows shell, or from eclipse pydev environment. what is the difference between running it from the shell, to running it from subprocess.call causing it to fail??

Thanks

---EDIT---

It seems I cant import encodings.idna either :

ImportError: No module named idna

which is very very weird... printing sys.path shows that c:\\Python27\\Lib is there, and the idna.py file is in the encodings directory... how can the import not work???

---ONE MORE EDIT---

Ive found one way around the problem - starting the IronPython script from a python shell - and everything works fine. I still have no Idea why invoking the python 2.7 interpreter from IronPython interferes with python 2.7 internal modules... the PYTHONPATH does include the path to those modules, so I would very much appreciate any insight to what is happening under the hood.

Answer 1

IronPython doesn't support idna encoding that is probably used by requests module.

idna encoding is available since Python 2.3:

>>> u"яндекс.рф".encode('idna')
b'xn--d1acpjx3f.xn--p1ai'
>>> b'xn--d1acpjx3f.xn--p1ai'.decode('idna')
'яндекс.рф'

To avoid ambiguity about what python is used to run your script, try specify the path to Python executable explicitly:

from subprocess import STDOUT, check_output as qx

output = qx([r"c:\Python27\python.exe", r"c:\path\to\script.py"], stderr=STDOUT)

As an alternative you could install pylauncher, to be able to specify a desired Python version for a Python script in its shebang line e.g., #!python2.7.