Segfault in pthreads on MIPS

Question

I'm trying to run my code on MIPS Linux machine (little-endian, without hardware floats):

# cat /proc/cpuinfo | grep model
cpu model               : MIPS 24Kc V5.5

My program works well on x86 and ARM systems but crashes with segmentation fault on MIPS, always in (or near) pthreads library function calls. I tried two toolchains (MTI GNU/Linux Toolchain MIPS32R2-MIPS32R5 from here and Sourcery CodeBench Lite 2016.05-8 from here) — details are slightly different, but segfault always occur.

Reading mail lists I found two reasons of pthreads segmentation faults on MIPS platform:

• probably wrong implementation of pthread_unlock_mutex() and pthread_wait_cond() in MIPS version of uClibc. But I use glibc (although the OS on device is uClibc-based);
• dynamic library load problem in pthreads; but my program is statically linked.

I wrote a very simple program:

#include <chrono>
#include <iostream>
#include <thread>

using namespace std;

void sayHello() {
    this_thread::sleep_for(chrono::seconds(2));
    cout << "Hello world!" << endl;
}

int main(int argc, char* argv[]) {
    thread t(sayHello);
    t.join();
    return 0;
}

It crashes on MIPS too. In the debugger with a breakpoin on t.join() I see the following:

(gdb) c
Continuing.

Breakpoint 1, __start () at ../sysdeps/mips/start.S:84
84      in ../sysdeps/mips/start.S
(gdb) disassemble 
Dump of assembler code for function __start:
   0x004059b0 <+0>:     lui     gp,0x5b
   0x004059b4 <+4>:     addiu   gp,gp,-30464
=> 0x004059b8 <+8>:     move    ra,zero
   0x004059bc <+12>:    lui     a0,0x40
   0x004059c0 <+16>:    addiu   a0,a0,21584
   0x004059c4 <+20>:    lw      a1,0(sp)
   0x004059c8 <+24>:    addiu   a2,sp,4
   0x004059cc <+28>:    li      at,-8
   0x004059d0 <+32>:    and     sp,sp,at
   0x004059d4 <+36>:    addiu   sp,sp,-32
   0x004059d8 <+40>:    lui     a3,0x4d
   0x004059dc <+44>:    addiu   a3,a3,-208
   0x004059e0 <+48>:    lui     t0,0x4d
   0x004059e4 <+52>:    addiu   t0,t0,4
   0x004059e8 <+56>:    sw      t0,16(sp)
   0x004059ec <+60>:    sw      v0,20(sp)
   0x004059f0 <+64>:    jal     0x4cf8a0 <__libc_start_main>
   0x004059f4 <+68>:    sw      sp,24(sp)
End of assembler dump.
… (some boring "n"-s here) …
(gdb) n
109     in ../sysdeps/mips/start.S
(gdb) disassemble 
Dump of assembler code for function __start:
   0x004059b0 <+0>:     lui     gp,0x5b
   0x004059b4 <+4>:     addiu   gp,gp,-30464
   0x004059b8 <+8>:     move    ra,zero
   0x004059bc <+12>:    lui     a0,0x40
   0x004059c0 <+16>:    addiu   a0,a0,21584
   0x004059c4 <+20>:    lw      a1,0(sp)
   0x004059c8 <+24>:    addiu   a2,sp,4
   0x004059cc <+28>:    li      at,-8
   0x004059d0 <+32>:    and     sp,sp,at
   0x004059d4 <+36>:    addiu   sp,sp,-32
   0x004059d8 <+40>:    lui     a3,0x4d
   0x004059dc <+44>:    addiu   a3,a3,-208
   0x004059e0 <+48>:    lui     t0,0x4d
   0x004059e4 <+52>:    addiu   t0,t0,4
   0x004059e8 <+56>:    sw      t0,16(sp)
   0x004059ec <+60>:    sw      v0,20(sp)
=> 0x004059f0 <+64>:    jal     0x4cf8a0 <__libc_start_main>
   0x004059f4 <+68>:    sw      sp,24(sp)
End of assembler dump.
(gdb) n
[New Thread 29775]

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) info registers 
          zero       at       v0       v1       a0       a1       a2       a3
 R0   00000000 00000000 00000016 005af490 2bc7d310 00000000 2bc7d378 00000000 
            t0       t1       t2       t3       t4       t5       t6       t7
 R8   00000000 00000000 00000000 82ee2000 00000000 00000000 00000001 74697773 
            s0       s1       s2       s3       s4       s5       s6       s7
 R16  7fbbf194 004450c4 00464200 004641f0 00460000 2b929478 0045742c 00000001 
            t8       t9       k0       k1       gp       sp       s8       ra
 R24  00000000 00000000 2bc847a0 00000000 005a8900 7fbbf158 7f894280 0041bd54 
        status       lo       hi badvaddr    cause       pc
      0100ff13 3b9aca00 00000000 00000000 50800008 00000000 
          fcsr      fir      hi1      lo1      hi2      lo2      hi3      lo3
      00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
        dspctl  restart
      00000000 00000000

A sample program without threads works well.

What can I do now? Have anyone similar experience with pthreads on MIPS?

Answer 1

Answering my own question.

Erik Alapää was right, build with uClibc resolved this issue. I still cannot understand why (program is statically linked).

By the way: I used glibc because was unable to build Boost.Asio with uClibc — pthreads library built with uClibc from my toolchain does not contain pthread_condattr_setclock() function. Fixed it with the following patch to Boost.Asio:

diff --git a/include/boost/asio/detail/impl/posix_event.ipp b/include/boost/asio/detail/impl/posix_event.ipp
index 4ff246f3..04f87a3a 100644
--- a/include/boost/asio/detail/impl/posix_event.ipp
+++ b/include/boost/asio/detail/impl/posix_event.ipp
@@ -33,10 +33,12 @@ posix_event::posix_event()
   : state_(0)
 {
 #if (defined(__MACH__) && defined(__APPLE__)) \
-      || (defined(__ANDROID__) && (__ANDROID_API__ < 21))
+      || (defined(__ANDROID__) && (__ANDROID_API__ < 21)) \
+      || (defined(__UCLIBC__))
   int error = ::pthread_cond_init(&cond_, 0);
 #else // (defined(__MACH__) && defined(__APPLE__))
       // || (defined(__ANDROID__) && (__ANDROID_API__ < 21))
+      // || (defined(__UCLIBC__))
   ::pthread_condattr_t attr;
   ::pthread_condattr_init(&attr);
   int error = ::pthread_condattr_setclock(&attr, CLOCK_MONOTONIC);
@@ -44,7 +46,7 @@ posix_event::posix_event()
     error = ::pthread_cond_init(&cond_, &attr);
 #endif // (defined(__MACH__) && defined(__APPLE__))
        // || (defined(__ANDROID__) && (__ANDROID_API__ < 21))
-
+       // || (defined(__UCLIBC__))
   boost::system::error_code ec(error,
       boost::asio::error::get_system_category());
   boost::asio::detail::throw_error(ec, "event");

Well, it's kind of a kludge, but it works.