Python receives SIGSEGV while loading C++ written extension module

Question

Right code example:

#include "Python.h"
#include <string>

extern const int someConstant;

void some_function()
{
  const char *begin = NULL;
  const char *end = NULL;

  std::string s(begin, end);
  const int v = someConstant;
}

static PyMethodDef _G_methods[] =
{
    {NULL, NULL, 0, NULL}        /* Sentinel */
};

PyMODINIT_FUNC initsf()
{
  PyObject *module;

  if (!(module = Py_InitModule("sf", _G_methods)))
  {
    return;
  }

  PyObject *pyerror = PyErr_NewException("fs.error", NULL, NULL);
  Py_INCREF(pyerror);


  PyModule_AddObject(module, "error", pyerror);
}

This is extension module draft. As simple as possible. It has an empty method table and initializer function copied from original docpage. It contains 2 (two) intentional errors:

• variable someConstant declared but never defined;

• function some_function defined, but never called;

If compiled and opened by dlopen/dlsym:

sf.so: undefined symbol: someConstant

as required. But if loaded by Python interpreter:

>>> from sf import *
Segmentation fault (core dumped)

and the most strange is python's backtrace dumped from core-file:

#0  0x00000bd6 in ?? ()
#1  0xb775c057 in char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) () from /usr/local/lib/python2.7/dist-packages/sf.so
#2  0xb6f9abb6 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#3  0xb6c3fe30 in pkgInitConfig(Configuration&) () from /usr/lib/i386-linux-gnu/libapt-pkg.so.4.12
#4  0xb6cf959e in ?? () from /usr/lib/python2.7/dist-packages/apt_pkg.so
#5  0x081949c1 in PyEval_EvalFrameEx ()
#6  0x0819af70 in PyEval_EvalCodeEx ()
#7  0x0819bb03 in PyImport_ExecCodeModuleEx ()
#8  0x0814bd40 in ?? ()
#9  0x080a38c2 in ?? ()
#10 0x0814c6d4 in ?? ()
#11 0x081031ae in ?? ()
...

It seems, Python's loader calls std::string constructor :-).

So, stack corrupted. It happens either while loading invalid module or while unloading it after error handled. It never happens if sample code is little changed. This behavior has been observed on Python 2.7.3/Linux Ubuntu 10/gcc 4.6.3 and definitely not shown on Python 2.7.1/FreeBSD 8.1/gcc 4.2.1.

Question:

1. Is it a Python's bug or my sample code has errors?

Answer 1

Let's look at that stack trace again

#0  0x00000bd6 in ?? ()
#1  0xb775c057 in char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) () from /usr/local/lib/python2.7/dist-packages/sf.so
#2  0xb6f9abb6 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&) () from /usr/lib/i386-linux-gnu/libstdc++.so.6
#3  0xb6c3fe30 in pkgInitConfig(Configuration&) () from /usr/lib/i386-linux-gnu/libapt-pkg.so.4.12
#4  0xb6cf959e in ?? () from /usr/lib/python2.7/dist-packages/apt_pkg.so

So a function in libapt-pkg.so calls a function in libstdc++.so which calls a function in your module.

Your functions are never getting called. However, your code uses std::string and instantiates some functions for std::string, those functions get included into your *.so, override those used by a completely different *.so, and crash for some reason I'm not entirely sure why.

My instincts tell me that you used gcc to create your *.so instead of g++. You won't get an error at link time because linking shared objects doesn't work that way. You won't get an error at load time because libstdc++ is coincidentally already loaded.

Are you using gcc or g++ to link? Try using g++.