/usr/bin/ld: cannot find -lpython-dev on Ubuntu. Compiling C program using PyObject

Question

I have the following C program which I need to integrate with Python:

#include <Python.h>

PyObject* get_details(){
    PyObject *details = PyDict_New();
    PyObject *key, *value;
    key = PyUnicode_FromString("full name");
    value = PyUnicode_FromString("Pete Graham");
    PyDict_SetItem(details, key, value);
    return details;
}

I can compile the program on OS X using:

gcc -I /Library/Frameworks/Python.framework/Versions/3.4/Headers \
-L /Library/Frameworks/Python.framework/Versions/3.4/lib \
-lpython3.4 get_person_dict.c

I get an error when I try and compile the program on Ubuntu /usr/bin/ld: cannot find -lpython3.4.

This is the command I'm using on Ubuntu I've tried various values for the python lib path.

gcc -I /usr/include/python3.4 \
-L /usr/local/lib \
-lpython3.4 get_person_dict.c

I installed python with sudo apt-get install python3.4-dev the version of Ubuntu trusty64 and is being used on Vagrant.

Answer 1

There are several things wrong.

As @JJHakala has advised you,

python3.4-config --libs

will tell you the libraries you need in order to link an executable with the Python 3.4 runtime, viz:

-lpython3.4m -lpthread -ldl  -lutil -lm

And

python3.4-config --ldflags

will tell you all the options you must pass to the linker to achieve the same end, viz:

-L/usr/lib/python3.4/config-3.4m-x86_64-linux-gnu \
-L/usr/lib -lpython3.4m -lpthread -ldl  -lutil -lm \
 -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions

So let's try putting those linker options into your gcc commandline instead of the linker options you've used. Your commandline was:

$ gcc -I /usr/include/python3.4 -L/usr/local/lib -lpython3.4 get_person_dict.c

with linker options:

-L/usr/local/lib -lpython3.4

So we'll try:

$ gcc -I /usr/include/python3.4 `python3.4-config --ldflags` get_person_dict.c

That doesn't work. The output is:

/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o: In function `_start':
/build/buildd/glibc-2.21/csu/../sysdeps/x86_64/start.S:114: undefined reference to `main'
/tmp/ccrczTMI.o: In function `get_details':
get_person_dict.c:(.text+0x9): undefined reference to `PyDict_New'
get_person_dict.c:(.text+0x17): undefined reference to `PyUnicode_FromString'
get_person_dict.c:(.text+0x25): undefined reference to `PyUnicode_FromString'
get_person_dict.c:(.text+0x40): undefined reference to `PyDict_SetItem'
collect2: error: ld returned 1 exit status

We have undefined references to all the Python 3.4 functions that are called in get_person_dict.c, and an undefined reference to main

The reason for that the undefined Python 3.4 references is that you were, and still are, passing the libraries to the linker before any file that calls functions defined in them. So the linker comes to each library on the commandline in turn and says to itself: Have I yet noted any calls to undefined functions that have definitions in this library? No. Then I don't need to link anything from this library. And passes on. After all the libraries have been ignored, it reaches your get_person_dict.c (or in reality, the temporary object file /tmp/ccrczTMI.o that the compiler has previously compiled from get_person_dict.c). There is finds undefined references PyDict_New, PyUnicode_FromString PyUnicode_FromString, PyDict_SetItem. But now there are no libraries left in which they might be defined. The libraries in which they are defined are water under the bridge, so the undefined references remain as linkage errors.

In the linkage sequence, files that need definitions must appear before the files that provide the definitions.

Well that's easily corrected:

$ gcc -I /usr/include/python3.4  get_person_dict.c `python3.4-config --ldflags`

Now the output is:

/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o: In function `_start':
/build/buildd/glibc-2.21/csu/../sysdeps/x86_64/start.S:114: undefined reference to `main'
collect2: error: ld returned 1 exit status

The linker has now found definitions for all the Python 3.4 functions. But it still says it can't find a definition for main.

The reason for that is the the C "program" you've posted isn't a program, it's just a definition of a function PyObject* get_details(). Every C program, by definition, has a main function, which is the first function called. For that reason, when you try to link some files and libraries into a C program, as you are trying to do, the linker automatically links startup code that calls main. But you have no main.

So you can't link your get_person_dict.c as a program, even if you get the linkage order right. You can compile it, without linking it:

$ gcc -I /usr/include/python3.4  -c -o get_person_dict.o get_person_dict.c

That gives you an object file get_person_dict.o which contains the compiled, but unlinked, definition of PyObject* get_details(). It will be identical to the temporary object file /tmp/ccrczTMI.o that gcc compiled for you without being told to in the failed compile-and-link attempt above.

But having compiled your get_person_dict.o like that, you can link it in a program, as long as you also link - after it - the other libraries that define the functions that it calls:

get_person_prog.c

#include <Python.h>
#include <stdio.h>

extern PyObject * get_details(); // Ought to be put in a header

int main(void)
{
    PyObject * pyobj = get_details();
    if (pyobj) {
        puts("Got the details");
    }
    return  0;
}

Compile that:

$ gcc -I /usr/include/python3.4  -c -o get_person_prog.o get_person_prog.c

Link your object files and libraries, in the right order, to make a program prog:

$ gcc -o prog get_person_prog.o get_person_dict.o `python3.4-config --ldflags`

Run:

$ ./prog
Got the details

Answer 2

In debian / jessie

python3.4-config --ldflags

tells what link parameters should be used, which should work on Ubuntu also. The command is in python3.4-dev package.

python3.4-config --libs

Tells the names of the libraries needed when linking.

The output

-L/usr/lib/python3.4/config-3.4m-x86_64-linux-gnu -L/usr/lib -lpython3.4m 
-lpthread -ldl -lutil -lm -Xlinker -export-dynamic 
-Wl,-O1 -Wl,-Bsymbolic-functions`

contains -lpython3.4m i.e not -lpython3.4 as in the question. Using it instead should help with this specific problem.