Switch from 32bit mode to 64 bit (long mode) on 64bit linux

Question

My program is in 32bit mode running on x86_64 CPU (64bit OS, ubuntu 8.04). Is it possible to switch to 64bit mode (long mode) in user mode temporarily? If so, how?

Background story: I'm writing a library linked with 32bit mode program, so it must be 32bit mode at start. However, I'd like to use faster x86_64 intructions for better performance. So I want to switch to 64bit mode do some pure computation (no OS interaction; no need 64bit addressing) and come back to 32bit before returning to caller.

I found there are some related but different questions. For example,

• run 32 bit code in 64 bit program
• run 64 bit code in 32 bit OS

My question is "run 64 bit code in 32 bit program, 64 bit OS"

Answer 1

Contrary to the other answers, I assert that in principle the short answer is YES. This is likely not supported officially in any way, but it appears to work. At the end of this answer I present a demo.

On Linux-x86_64, a 32 bit (and X32 too, according to GDB sources) process gets CS register equal to 0x23 — a selector of 32-bit ring 3 code segment defined in GDT (its base is 0). And 64 bit processes get another selector: 0x33 — a selector of long mode (i.e. 64 bit) ring 3 code segment (bases for ES, CS, SS, DS are treated unconditionally as zeros in 64 bit mode). Thus if we do far jump, far call or something similar with target segment selector of 0x33, we'll load the corresponding descriptor to the shadow part of CS and will end up in a 64 bit segment.

The demo at the bottom of this answer uses jmp far instruction to jump to 64 bit code. Note that I've chosen a special constant to load into rax, so that for 32 bit code that instruction looks like

dec eax
mov eax, 0xfafafafa
ud2
cli ; these two are unnecessary, but leaving them here for fun :)
hlt

This must fail if we execute it having 32 bit descriptor in CS shadow part (will raise SIGILL on ud2 instruction).

Now here's the demo (compile it with fasm).

format ELF executable
segment readable executable

SYS_EXIT_32BIT=1
SYS_EXIT_64BIT=60
SYS_WRITE=4
STDERR=2

entry $
    mov ax,cs
    cmp ax,0x23 ; 32 bit process on 64 bit kernel has this selector in CS
    jne kernelIs32Bit
    jmp 0x33:start64 ; switch to 64-bit segment
start64:
use64
    mov rax, qword 0xf4fa0b0ffafafafa ; would crash inside this if executed as 32 bit code
    xor rdi,rdi
    mov eax, SYS_EXIT_64BIT
    syscall
    ud2

use32
kernelIs32Bit:
    mov edx, msgLen
    mov ecx, msg
    mov ebx, STDERR
    mov eax, SYS_WRITE
    int 0x80
    dec ebx
    mov eax, SYS_EXIT_32BIT
    int 0x80
msg:
    db "Kernel appears to be 32 bit, can't jump to long mode segment",10
msgLen = $-msg