GNU Arm Embedded Toolchain: undefined reference to `__sync_synchronize'

Question

I am trying to update the tool-chain for a bare metal embedded project. We are building on windows an up to now we have been using version 5.4.1 20160609 (release) [ARM/embedded-5-branch revision 237715].

Now I am trying version 9.3.1 20200408 (release) (9-2020-q2-update), but I have an issue with an undefined symbol while linking: __sync_synchronize is reported as missing and I have no idea from which source this symbol should be resolved. Do I have to link a library that I am missing? Should I give different flags to the compiler so that it generates the code for that function?

Below is a sample that compiles and links just fine with the old tool-chain, but fails with the new one. In both cases this command line was used:

arm-none-eabi-g++ -mthumb -specs=nosys.specs sample.cpp

The exact failure message is:

c:/projects/cpt_tools/gcc-arm-none-eabi/9.3.1/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\noone\AppData\Local\Temp\ccSZkMXN.o: in function 'use_static_inst(int)': sample.cpp:(.text+0xc): undefined reference to '__sync_synchronize' collect2.exe: error: ld returned 1 exit status

BTW: There is no difference when I run my test on Linux:

noone@nowhere:~/gcc-arm-none-eabi-9-2020-q2-update/bin$ ./arm-none-eabi-g++ -mthumb -specs=nosys.specs sample.cpp /media/persistent_storage/home/rmatano/gcc-arm-none-eabi-9-2020-q2-update/bin/../lib/gcc/arm-none-eabi/9.3.1/../../../../arm-none-eabi/bin/ld: /tmp/ccltslyj.o: in function 'use_static_inst(int)': />sample.cpp:(.text+0xc): undefined reference to '__sync_synchronize' collect2: error: ld returned 1 exit status

I found this question GCC Linaro cross compile fails on linker step on a Windows host that seems to deal with the same problem. But I cannot use the suggested solution of specifying -mcpu=cortex-a9 since my code should run on a quite old hardware ('-cpu=arm926ej-s').

Here is the content of sample.cpp:

// compile with 'arm-none-eabi-g++.exe -mthumb -specs=nosys.specs sample.cpp'
//
// result: in function `use_static_inst(int)':
// sample.cpp:(.text+0x18): undefined reference to `__sync_synchronize'

class A
{
int m_i;

public:

    A(int i) : m_i(i)
    {
    }

    int value(int x)
    {
        return m_i + x;
    }
};


int use_static_inst(int x)
{
    // in preparation for calling the ctor of this static instance
    // the compiler generates a call to __sync_synchronize
    static A a(0);

    return a.value(x);
}

int main(int argc, char* argv[])
{
    return use_static_inst(argc);
}

Answer 1

Disclaimer: I am not within my area of expertise here.

The missing function is implementing a Memory Barrier. This is needed in multi-core CPU such as the Cortex-A9, but this is unlikely that this is needed on your arm926ej-s. Since the toolchain you are using is explicitely targeting Cortex-M and Cortex-R CPUs, I would not be surprised if the new toolchain would not have been tested a lot (or at all) against ARM9 CPUs.

My first comment would be that you should IMHO not let the compiler use its default settings, and explicit the exact ARM architecture and CPU you are targeting - please note that this does not solve your issue, the linker error will still be present:

arm-none-eabi-g++ -march=armv5tej -mcpu=arm926ej-s -mthumb -specs=nosys.specs sample.cpp 

An interesting thing to note is that compiling using -marm does not trigger the linker error:

arm-none-eabi-g++ -march=armv5tej -mcpu=arm926ej-s -marm -specs=nosys.specs sample.cpp 

In this case, the code generated for __sync_synchronize() would be:

00011014 <__sync_synchronize>:
   11014:   f44f 637a   mov.w   r3, #4000   ; 0xfa0
   11018:   f6cf 73ff   movt    r3, #65535  ; 0xffff
   1101c:   4718        bx  r3
   1101e:   bf00        nop

My understanding is that the would result in the program branching to address 0xffff0fa0, which seems a little bit odd: we are using an arm-none-eabi toolchain, and it seems the generated code is trying to trigger the execution of a Linux kuser_memory_barrier- see documentation here. This may be that it is the equivalent of calling __sync_synchronize() in a Linux environment. This is IMHO an undetected bug in the toolchain.

Now, __sync_synchronize() is a gcc builtin and does issue a full memory barrier. When targeting, say, a Cortex-A9, it will generate a Data Memory Barrier dmb ish instruction. This instruction is required because the Cortex-A9 does support Out-Of-Order execution, and therefore programs may need to execute a full memory barrier at some points, more specifically if more that one core is present.

This is interesting to note that the code used for implementing the kuser_memory_barrier in Linux 5.9.6 (file arch/arm64/kernel/kuser32.S) is using dmb ish as well:

__kuser_memory_barrier:         // 0xffff0fa0
    .inst   0xf57ff05b      //  dmb     ish
    .inst   0xe12fff1e      //  bx      lr

That is, the underlying hardware is likely required to provide a minimal support for the builtin implementation if a Data Memory Barrier is needed.

Since it looks like the arm926ej-s has only the Instruction Memory Barrier IMB instruction available, this may just be that the arm926ej-s does not support Out-Of-Order execution, in this case calling __sync_synchronize() would not be required, and you may try to provide an empty/do nothing implementation of __sync_synchronize() and still be safe. You need to be sure on whether this is the case or not.

Note that the the arm926ej-s Reference Manual does only mention Out-Of-Order when it comes to Tightly-Coupled Memory Interface transactions, but again, you need to look in depth. You should probably look at the code generated for __cxa_guard_acquire() and __cxa_guard_release() as well in order to fully understand how they relate to your hardware. You probably don't want to have to debug weird problems in a multi-threaded application.

By looking at their implementation in gcc-arm-none-eabi-9-2020-q2-update/arm-none-eabi/lib/libstdc++_nano.a for example, I don't see any specific instructions that may prevent their use on your target, but again I am not familiar with the arm926ej-s instruction set, and you are the one who should ultimately make the call:

Disassembly of section .text.__cxa_guard_acquire:

00000000 <__cxa_guard_acquire>:
   0:   e5902000    ldr r2, [r0]
   4:   e3120001    tst r2, #1
   8:   e1a03000    mov r3, r0
   c:   1a000006    bne 2c <__cxa_guard_acquire+0x2c>
  10:   e5d02001    ldrb    r2, [r0, #1]
  14:   e3520000    cmp r2, #0
  18:   0a000000    beq 20 <__cxa_guard_acquire+0x20>
  1c:   e7f000f0    udf #0
  20:   e3a00001    mov r0, #1
  24:   e5c30001    strb    r0, [r3, #1]
  28:   e12fff1e    bx  lr
  2c:   e3a00000    mov r0, #0
  30:   e12fff1e    bx  lr

Disassembly of section .text.__cxa_guard_abort:

00000000 <__cxa_guard_abort>:
   0:   e3a03000    mov r3, #0
   4:   e5c03001    strb    r3, [r0, #1]
   8:   e12fff1e    bx  lr

Disassembly of section .text.__cxa_guard_release:

00000000 <__cxa_guard_release>:
   0:   e3a03001    mov r3, #1
   4:   e5803000    str r3, [r0]
   8:   e12fff1e    bx  lr

guard_error.o:     file format elf32-littlearm

Those extra precautions are probably needed since you are using a C++ compiler released in 2019 on a family of CPU that is almost 20 years old.

Answer 2

I had the same error attempting to link for a Cortex-R4 using gcc-arm-none-eabi-6-2017-q1-update. The linker error was:

Invoking: GNU Linker
"C:/ti/ccs1040/ccs/tools/compiler/gcc-arm-none-eabi-6-2017-q1-update/bin/arm-none-eabi-gcc-6.3.1.exe" -Og -g -gdwarf-3 -gstrict-dwarf -Wall -specs="nosys.specs" -mfloat-abi=hard -Wl,-Map,"RM46L850_GCC_halcogen_cpp.map" -o"RM46L850_GCC_halcogen_cpp.out" "./source/cpp_test.o" "./source/dabort.o" "./source/errata_SSWF021_45.o" "./source/esm.o" "./source/notification.o" "./source/pinmux.o" "./source/sci.o" "./source/sys_core.o" "./source/sys_dma.o" "./source/sys_intvecs.o" "./source/sys_main.o" "./source/sys_mpu.o" "./source/sys_pcr.o" "./source/sys_phantom.o" "./source/sys_pmm.o" "./source/sys_pmu.o" "./source/sys_selftest.o" "./source/sys_startup.o" "./source/sys_vim.o" "./source/system.o" -Wl,-T"../source/sys_link.ld"  -Wl,--start-group -lc -lstdc++ -Wl,--end-group 
makefile:163: recipe for target 'RM46L850_GCC_halcogen_cpp.out' failed
c:/ti/ccs1040/ccs/tools/compiler/gcc-arm-none-eabi-6-2017-q1-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/lib/hard\libstdc++.a(locale_init.o): In function `(anonymous namespace)::get_locale_mutex()':
locale_init.cc:(.text._ZN12_GLOBAL__N_116get_locale_mutexEv+0xc): undefined reference to `__sync_synchronize'
c:/ti/ccs1040/ccs/tools/compiler/gcc-arm-none-eabi-6-2017-q1-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/lib/hard\libstdc++.a(locale.o): In function `std::locale::_Impl::_M_install_cache(std::locale::facet const*, unsigned int)':
locale.cc:(.text._ZNSt6locale5_Impl16_M_install_cacheEPKNS_5facetEj+0x18): undefined reference to `__sync_synchronize'
c:/ti/ccs1040/ccs/tools/compiler/gcc-arm-none-eabi-6-2017-q1-update/bin/../lib/gcc/arm-none-eabi/6.3.1/../../../../arm-none-eabi/lib/hard\libstdc++.a(future.o): In function `std::future_category()':
future.cc:(.text._ZSt15future_categoryv+0xc): undefined reference to `__sync_synchronize'
collect2.exe: error: ld returned 1 exit status

In my case the issue was that the TI Eclipse based Code Composer Studio which was creating the Makefiles wasn't giving the same Runtime options to the linker to select the multi-lib library as was passed to the compiler.

The compiler was given -mcpu=cortex-r4 -mfloat-abi=hard -mfpu=vfpv3-d16 but the linker was only given -mfloat-abi=hard. This resulted in the linker selecting the hard library which had the undefined reference to __sync_synchronize

Once the linker was given the same -mcpu=cortex-r4 -mfloat-abi=hard -mfpu=vfpv3-d16 options as the compiler then selected the thumb/v7-ar/fpv3/hard multi-lib libraries and the link was successful.

For reference, in https://e2e.ti.com/support/tools/code-composer-studio-group/ccs/f/code-composer-studio-forum/1023679/ccs-10-4-0-00006-for-the-gcc-arm-compiler-how-does-ccs-determine-how-many-of-the-compiler-runtime-options-to-pass-to-the-linker queried why CCS didn't pass a consistent set of Runtime options to the linker and compiler.