GCC arm instruction mode when compiling in thumb mode

Question

I'm wondering how GCC configured using --with-mode=thumb handles compiling/assembling code that makes use of ARM mode sections if the -marm flag is not specified. That is:

• GCC is compiled with --with-mode=thumb
• A program is compiled without -marm (defaults to thumb mode)
• An assembly section of that program uses ARM mode

I tried compiling a small test program on Raspberry Pi 4 with Ubuntu 18.04.4 kernel 5.3.0-1018-raspi2 and noticed that the .arm section is being executed in 16-bit thumb instruction mode which prompted me to investigate this. This naturally causes a segmentation fault as the program counter is increment by 2 bytes instead of 4.

Here's what gdb in layout asm mode says when my program branches into the .arm assembly code and after I perform a single stepi command:

0x400900 <asm_maxfilter>        push   {r4, lr}
0x400904 <asm_maxfilter+4>      mov    r3, #0
0x400908 <filter_loop>          vld1.8 {d0-d1}, [r0]

pc 0x400902 0x400902 <asm_maxfilter+2>
^ The program counter is between instructions

My code is as follows:

#include <arm_neon.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>

void asm_maxfilter(unsigned char* upbuffer, unsigned char* longterm_buffer, int grid_size);

int main(int argc, char** argv) {

    const int pixels_per = 16;
    const int grid_reso = 256;
    const int grid_size = grid_reso * grid_reso;
    const int remainder = grid_size % pixels_per;
    const int work_count = grid_size - remainder;

    unsigned char* longterm_up = (unsigned char*)malloc(grid_reso * grid_reso);
    memset(longterm_up, 0, grid_reso * grid_reso);

    unsigned char* up_buffers[60];
    int u;
    int i;

    for (u = 0; u < 60; ++u) {
        up_buffers[u] = (unsigned char*)malloc(grid_reso * grid_reso);

        if (up_buffers[u] == NULL) {
            fprintf(stderr, "Failed mallocing\n");
            return 1;
        }

        memset(up_buffers[u], 0, grid_reso * grid_reso);
    }

    for (u = 0; u < 60; ++u) {

        asm_maxfilter(up_buffers[u], longterm_up, work_count);

        // non-SIMD version handles the remainder that did not fit in NEON registers
        for (i = grid_size - remainder; i < grid_size; ++i) {
            if (longterm_up[i] < up_buffers[u][i]) {
                longterm_up[i] = up_buffers[u][i];
            }
        }
    }

    for (u = 0; u < 60; ++u) {
        free(up_buffers[u]);
    }

    free(longterm_up);

    return 0;
}

Assembly:

@ ARM NEON version of a max filter. Performs the following operation:
@
@ for (int i = 0; i < buf_size; ++i) {
@   if (buf_b[i] < buf_a[i]) {
@       buf_b[i] = buf_a[i];
@   }
@ }

.arm
.section .text
.align 4
.globl asm_maxfilter

@ parameters
@ r0: buf_a
@ r1: buf_b
@ r2: buf_size, multiple of 16
asm_maxfilter:

    @ Store register states in stack. They must be restored before returning
    push { r4, lr }

    @ Reset counter
    mov r3, #0

    filter_loop:

        @ Load 16 bytes into vectors
        vld1.u8 {q0}, [r0]
        vld1.u8 {q1}, [r1]

        @ Find greater values in each vector
        vcgt.u8 q2, q0, q1

        @ Bitselect the greater value into q2
        vbsl.u8 q2, q0, q1

        @ Store the larger value in output buffer
        vst1.u8 {q2}, [r1]

        @ Increment counter by 16
        add r3, r3, #16

        @ Increment pointers
        add r0, r0, #16
        add r1, r1, #16

        @ Check if loop is done
        cmp r3, r2
        blt filter_loop

    @ Restore registers to their original state
    pop { r4, lr }

    @ lr register contains return address
    bx lr

.end

The code is compiled using:

gcc -Wall -Wpedantic -O0 -g -march=armv8-a -mfloat-abi=hard -mtune=cortex-a72 -mfpu=neon -c -o main.o main.c
gcc -Wall -Wpedantic -O0 -g -march=armv8-a -mfloat-abi=hard -mtune=cortex-a72 -mfpu=neon -o neon_test ./main.o ./asm_test.s

Based on what the ARM documentation says, if the processor needs to switch between thumb/arm the program should perform a branch using the BLX or BX instruction:

https://developer.arm.com/docs/100076/0100/instruction-set-overview/overview-of-aarch32-state/changing-between-a32-and-t32-instruction-set-states

Quoting:

To direct armasm to generate A32 or T32 instruction encodings, you must set the assembler mode using an ARM or THUMB directive. Assembly code using CODE32 and CODE16 directives can still be assembled, but Arm recommends you use the ARM and THUMB directives for new code.

These directives do not change the instruction set state of the processor. To do this, you must use an appropriate instruction, for example BX or BLX to change between A32 and T32 states when performing a branch.

After disassembling my program, I noticed this mode switching is not done. Is this something that the programmer must do themselves in their assembly code (even though the branching happens from the C code), or should the compiler/assembler handle this?

I also tried specifying __attribute__((target("arm"))) in the C file function declaration, that is:

__attribute__((target("arm")))
void asm_maxfilter(unsigned char* upbuffer, unsigned char* longterm_buffer, int grid_size);

However, this didn't seem to change anything. Everything works correctly as soon as I compile with -marm or use GCC that doesn't have --with-mode=thumb

Answer 1

As suggested by old_timer in a comment, the problem was that the assembly source code did not include .type asm_maxfilter, %function before the label. The working assembly code begins as follows:

.arm
.section .text
.align 4
.globl asm_maxfilter

.type asm_maxfilter, %function
asm_maxfilter:

    @ Store register states in stack. They must be restored before returning
    push { r4, lr }

    @ Reset counter
    mov r3, #0
    ...

If the situation was reversed (ARM mode program using a thumb function), then instead of .type asm_maxfilter, %function the type should have been .thumb_func.

As per Jester's response, I noticed that the C code object file indeed has a R_ARM_THM_CALL relocation segment, but without using the .type macro, the branch instruction was not replaced by a bx instruction.

If one implements an ARM function in a C file by using __attribute__((target("arm"))) without external assembly, ie:

#include <stdio.h>
#include <stdlib.h>

__attribute__((target("arm")))
void foo(int a) {
    int b = 6*a;
    fprintf(stderr, "%d\n", b*5);
}

int main(int argc, char** argv) {
    int asd = atoi(argv[1]);
    foo(asd);
    return 0;
}

Then one can obseve a blx instruction being correctly used in the generated binary. The problem I had is only a concern if one uses assembly code in a separate file that does not pass through a compiler.