High Sierra + Python + Postgresql error: Illegal instruction: 4

Question

I can connect to Postgres via the command line no problem, however if I try to connect via Python using the psycopg2 module I get the error below. Interestingly I have just tried connecting with PSeqal.app and it crashes with the same error.

Environment:

• MacOS: 10.13.3
• Xcode 9.2
• Python: 3.6.4
  • Connection module: psycopg2
• Postgresql: 10.3

Script:

Here is my simple connection script which is trying to connect via Python to Postgresql:

def connect(self, params):
    """ Connect to the PostgreSQL database server """

    conn = None
    try:
        # connect to the PostgreSQL server
        logging.info('Connecting to the PostgreSQL database...')
        conn = psycopg2.connect(**params)

        # create a cursor
        self.cursor = conn.cursor()

        except (Exception, psycopg2.DatabaseError) as error:
            print(error)
        finally:
            if conn is not None:
                conn.close()
                logging.warning('Database connection closed.')

Error:

Script output:

$ python3 testdb.py
Illegal instruction: 4

Apple error (partial):

Process:               Python [79534]
Path:                  /usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.6.4 (3.6.4)
Code Type:             X86-64 (Native)
Parent Process:        bash [657]
Responsible:           Python [79534]
User ID:               501

Date/Time:             2018-03-02 15:30:27.642 +1300
OS Version:            Mac OS X 10.13.3 (17D102)
Report Version:        12
Anonymous UUID:        xx


Time Awake Since Boot: 21000 seconds

System Integrity Protection: disabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [0]

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 
0   libpq.5.10.dylib                0x00000001118e8d27 initPQExpBuffer + 32
1   libpq.5.10.dylib                0x00000001118dba93 PQconninfoParse + 43
2   _psycopg.cpython-36m-darwin.so  0x0000000111897cf1 psyco_parse_dsn + 113
3   org.python.python               0x000000010fd4681a _PyCFunction_FastCallDict + 463
4   org.python.python               0x000000010fdaad8e call_function + 489
5   org.python.python               0x000000010fda3c43 _PyEval_EvalFrameDefault + 4811
6   org.python.python               0x000000010fdab4f0 _PyEval_EvalCodeWithName + 1719
7   org.python.python               0x000000010fda293d PyEval_EvalCodeEx + 57
8   org.python.python               0x000000010fd2ec76 function_call + 339
9   org.python.python               0x000000010fd0e9f2 PyObject_Call + 101
10  org.python.python               0x000000010fda3eac _PyEval_EvalFrameDefault + 5428
11  org.python.python               0x000000010fdab4f0 _PyEval_EvalCodeWithName + 1719
12  org.python.python               0x000000010fda293d PyEval_EvalCodeEx + 57
13  org.python.python               0x000000010fd2ec76 function_call + 339
14  org.python.python               0x000000010fd0e9f2 PyObject_Call + 101
15  org.python.python               0x000000010fda3eac _PyEval_EvalFrameDefault + 5428
16  org.python.python               0x000000010fdabe93 _PyFunction_FastCall + 121
17  org.python.python               0x000000010fdaad65 call_function + 448
18  org.python.python               0x000000010fda3c43 _PyEval_EvalFrameDefault + 4811
19  org.python.python               0x000000010fdabe93 _PyFunction_FastCall + 121
20  org.python.python               0x000000010fd0eb71 _PyObject_FastCallDict + 196
21  org.python.python               0x000000010fd0ec94 _PyObject_Call_Prepend + 156
22  org.python.python               0x000000010fd0e9f2 PyObject_Call + 101
23  org.python.python               0x000000010fd598ae slot_tp_init + 57
24  org.python.python               0x000000010fd5683c type_call + 184
25  org.python.python               0x000000010fd0eb3c _PyObject_FastCallDict + 143
26  org.python.python               0x000000010fdaad5e call_function + 441
27  org.python.python               0x000000010fda3c43 _PyEval_EvalFrameDefault + 4811
28  org.python.python               0x000000010fdabe93 _PyFunction_FastCall + 121
29  org.python.python               0x000000010fd0eb71 _PyObject_FastCallDict + 196
30  org.python.python               0x000000010fd0ec94 _PyObject_Call_Prepend + 156
31  org.python.python               0x000000010fd0e9f2 PyObject_Call + 101
32  org.python.python               0x000000010fd598ae slot_tp_init + 57
33  org.python.python               0x000000010fd5683c type_call + 184
34  org.python.python               0x000000010fd0eb3c _PyObject_FastCallDict + 143
35  org.python.python               0x000000010fdaad5e call_function + 441
36  org.python.python               0x000000010fda3c43 _PyEval_EvalFrameDefault + 4811
37  org.python.python               0x000000010fdab4f0 _PyEval_EvalCodeWithName + 1719
38  org.python.python               0x000000010fda28fe PyEval_EvalCode + 42
39  org.python.python               0x000000010fdcb24e run_mod + 54
40  org.python.python               0x000000010fdca26f PyRun_FileExFlags + 160
41  org.python.python               0x000000010fdc994c PyRun_SimpleFileExFlags + 285
42  org.python.python               0x000000010fddd770 Py_Main + 3484
43  org.python.python               0x000000010fd01e1d 0x10fd00000 + 7709
44  libdyld.dylib                   0x00007fff6c260115 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x00007f86c3db9200  rbx: 0x00007ffedfefe3a0  rcx: 0x0000000000000100  rdx: 0x0000000000010000
rdi: 0x0000000000000b93  rsi: 0x00000000ffff0001  rbp: 0x00007ffedfefe390  rsp: 0x00007ffedfefe380
r8: 0x000000006c3db930   r9: 0x000000000000000f  r10: 0x00000000ffff0000  r11: 0x00007f86c3d00000
r12: 0x0000000110665738  r13: 0x0000000111c68510  r14: 0x0000000111cb80b0  r15: 0x00007ffedfefe3a0
rip: 0x00000001118e8d27  rfl: 0x0000000000010206  cr2: 0x00000001118ef90e

Logical CPU:     2
Error Code:      0x00000000
Trap Number:     6

What I have tried so far:

• Upgraded Python to 3.6.4 from 3.5.1
  • Rebuilt all dependent libraries including psycopg2 to make sure they are High Sierra compatible (I hope)
• Reinstalled Postgres from scratch
• Upgraded Xcode + tools

So I'm at a bit of a loss as to what to try next? Has anyone else run into this? Any pointers to where to look? Anything that helps me get this working would be greatly appreciated thanks!

Answer 1

So it looked like the psycopg2 library was the culprit and this fixed it for me:

$pip3 uninstall psycopg2
...
Successfully uninstalled psycopg2-binary-2.7.4

$pip3 install psycopg2-binary --no-binary :all:
Collecting psycopg2-binary Downloading psycopg2-binary-2.7.4.tar.gz (426kB) 100% |---| 430kB 649kB/s Skipping bdist_wheel for psycopg2-binary, due to binaries being disabled for it. Installing collected packages: psycopg2-binary Running setup.py install for psycopg2-binary ... done
Successfully installed psycopg2-binary-2.7.4

$python3 test.py
Connecting to the PostgreSQL database...
All good.
Database connection closed.

The reason I believe is something to do with the fact I was trying to build psycopg2 on a 2010 era MBP specifically and it has a compatibility issue with the hardware - probably the 2.66 GHz Intel Core i7 processor this machine has.

Thank you @Laurenz Albe and @joop for quick replies, pointers and insights, you helped narrow me down to this answer which I hope will stay stable for the duration of this job ;) Cheers!

Answer 2

According to the stack trace, that error occurs in the PostgreSQL client, in this harmless piece of code:

/*
 * initPQExpBuffer
 *
 * Initialize a PQExpBufferData struct (with previously undefined contents)
 * to describe an empty string.
 */
void
initPQExpBuffer(PQExpBuffer str)
{
    str->data = (char *) malloc(INITIAL_EXPBUFFER_SIZE);
    if (str->data == NULL)
    {
        str->data = (char *) oom_buffer;    /* see comment above */
        str->maxlen = 0;
        str->len = 0;
    }
    else
    {
        str->maxlen = INITIAL_EXPBUFFER_SIZE;
        str->len = 0;
        str->data[0] = '\0';
    }
}

I found this and this question dealing with similar problems.

Illegal instruction should mean that execution encountered an instruction code that does not exist, which is either a compiler bug or means that the code was built for a processor with a different instruction set.

The problem is likely somewhere in the procedure with which PostgreSQL was built. You should bring this complaint to the packagers who built the binaries you are using.