Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How are symbols contained in the libpythonX.X linked to numpy extension dynamic libraries?

I'm currently having a problem where I'm able to use and import numpy in an interpreter environment, but I'm unable to import or use numpy from python embedded in C/C++. So I'm curious to how numpy extension libraries, specifically

numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so

is linked to the standard python package symbols (PyExc_UserWarning symbol specifically). My current error output says that PyExc_UserWarning is undefined. This symbol exists in libpythonX.Y.m.so as I confirmed using the nm command. I ran

ldd multiarray.cpython-35m-x86_64-linux-gnu.so

and got the following output:

Console Output

It does not seem to me that this library is linked to any dynamic libraries that should contain that symbol. How does numpy's multiarray.cpython-35m-x86_64-linux-gnu.so usually find that symbol or the variations of multiarray find that symbol?

Thank you for taking your time from your day reading this question. Any thoughts, suggestion, or answers are appreciated!

The original question is located here. This is a sub-question of the original question. The reason why I'm asking this question is because I'm suspecting that this shared library might be linked to the wrong location, and this specific shared library is only used when calling python through the python C/C++ interfaces.

System Specs + Problem information

  • Ubuntu 16.04, 64 bit
  • Compiled Python 3.5.5 with enabled-shared
  • Installed numpy-1.14.2 using pip 9.0.0 using the pip3.5 install numpy command

Edit 4/16/18:

modified some terminology that was unclear.

Edit 4/17/18:

I found an answer to the original problem; however, this question and the original one are still open, because an answer to this question could lead to a better answer for the original problem.

like image 541
skincell Avatar asked Apr 16 '18 21:04

skincell


1 Answers

multiarray.cpython-35m-x86_64-linux-gnu.so was built without explicit link with python's dynamic library, that why you couldn't see the libpythonx.x.x by using ldd.

If you use nm to check this so, you will see the symbol PyExc_UserWarning is undefined.

So when numpy load this so by using dlopen, it will try to resolve this undefined symbol. I didn't find any document explain the rule how libdl.so resolve the undefined symbol. But according to my test, when you try to open a shared library use dlopen with flag RTLD_NOW, it will search dependent shared library of main program for the undefined symbol.

That could explain why python could use it without error, because python binary is linked with libpython.x.x.so.

like image 101
2power10 Avatar answered Oct 31 '22 18:10

2power10