Performance implications of context switches for 64-bit segment bases

Question

I'm confused by the wording in the manual page for arch_prctl(2). Specifically, it states:

Context switches for 64-bit segment bases are rather expensive. It may be a faster alternative to set a 32-bit base using a segment selector by setting up an LDT with modify_ldt(2) or using the set_thread_area(2) system call in kernel 2.5 or later. arch_prctl() is only needed when you want to set bases that are larger than 4GB. Memory in the first 2GB of address space can be allocated by using mmap(2) with the MAP_32BIT flag.

Does this mean that context switches for a process that uses this system call will receive a performance penalty or what are the exact implications?

After looking through the source of the Linux kernel, it appears that for addresses that are <4 GiB use the LDT, while >4 GiB addresses use a model-specific register.

From do_arch_prctl:

case ARCH_SET_FS:
        /* handle small bases via the GDT because that's faster to
           switch. */
        if (addr <= 0xffffffff) {
                set_32bit_tls(task, FS_TLS, addr);
                if (doit) {
                        load_TLS(&task->thread, cpu);
                        loadsegment(fs, FS_TLS_SEL);
                }
                task->thread.fsindex = FS_TLS_SEL;
                task->thread.fs = 0;
        } else {
                task->thread.fsindex = 0;
                task->thread.fs = addr;
                if (doit) {
                        /* set the selector to 0 to not confuse
                           __switch_to */
                        loadsegment(fs, 0);
                        ret = wrmsrl_safe(MSR_FS_BASE, addr);
                }
        }
        put_cpu();
        break;

How can using the GDT be faster than writing to a register? Also, I assume the price of updating FS and GS is only paid when switching between processes, meaning there's no additional cost for entering the kernel via a system call when no other process is scheduled to run?

Answer 1

Wow, this was asked in December and nobody answered it? Some of this you may already know, and I apologize if so.

It's just because the steps for doing the wrmsr are slow. It's easier and faster to just load the segment registers when task switching.

On very modern Intel processors, the addition of the "rdfsbase", "wrfsbase", "rdgsbase" and "wrgsbase" instructions allow direct access to the base registers of FS and GS with far less difficulty than before. In fact, the kernel can allow their use from user mode if it likes. You might want to check whether modern Linux kernels take advantage of wrfsbase to make allocating the TLS area below 4 GB unnecessary.

I don't know how it is on Linux, but Windows NT starting with Windows 7 has user-mode thread scheduling as an optional feature to application developers. As in Linux and Mac OS X, Windows implements thread-local storage on x86 using the base address of the segment registers (GS in x86-64 Windows). This feature is similar to fibers, except that a program can switch out its own thread context with another in a way that is also recognized by the kernel.

User-mode scheduling in Windows is implemented by creating an LDT with segments pointing to the TLS blocks (called "Thread Environment Blocks", or TEBs, in Windows) for each schedulable thread. User mode can then switch threads by reloading the GS base in addition to the context switch. This requires the TEBs to be below 2^32 as in Linux's arch_prctl's performance note - otherwise, user-mode scheduling would require a call to the NT kernel every time it switched to a different thread in order to do a wrmsr, defeating the whole point of user-mode scheduling.

In Windows 8.1, support for wrgsbase was added, and it is enabled for user-mode as well. User-mode scheduling in 8.1 uses wrgsbase instead of GS segment reloads and the LDT if the CPU has it.