Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does the BIOS entry point start with a WBINVD instruction?

I'm investigating the BIOS code in my machine (x86_64 Linux, IvyBridge). I use the following procedure to dump the BIOS code:

$ sudo cat /proc/iomem | grep ROM
  000f0000-000fffff : System ROM
$ sudo dd if=/dev/mem of=bios.dump bs=1M count=1

Then I use radare2 to read and disassemble the binary dump:

$ r2 -b 16 bios.dump 
[0000:0000]> s 0xffff0
[f000:fff0]> pd 3
        :   f000:fff0      0f09           wbinvd
        `=< f000:fff2      e927f5         jmp 0xff51c
            f000:fff5      0000           add byte [bx + si], al

I know x86 processor initialization always starts with a 16-bit 8086 environment, and the first instruction to be executed is at f000:fff0, i.e. 0xffff0. So I go to that location and disassemble the code.

To my surprise, the first instruction is WBINVD, whose functionality is to invalidate the cache, which seems to be irrelevant when the processor is powered on or reset. I would expect the first instruction to be simply a jmp to a lower memory address.

Why is there a WBINVD before jmp?

I've already searched the relevant portion of the Intel manuals, Volume 3 Chapter 9 Processor Management and Initialization, but it doesn't mention anything about WBINVD. I also searched some online resources but didn't find any explanation.

Edit for more info:

After following the jmp instruction to 0xff51c, the code is more interesting; it's doing a self-check:

[f000:f51c]> pd
            f000:f51c      dbe3           fninit
            f000:f51e      0f6ec0         movd mm0, eax
            f000:f521      6631c0         xor eax, eax
            f000:f524      8ec0           mov es, ax
            f000:f526      8cc8           mov ax, cs
            f000:f528      8ed8           mov ds, ax
            f000:f52a      b800f0         mov ax, 0xf000
            f000:f52d      8ec0           mov es, ax
            f000:f52f      6726a0f0ff00.  mov al, byte es:[0xfff0]     ; [0xfff0:1]=0
            f000:f536      3cea           cmp al, 0xea
        ,=< f000:f538      750f           jne 0xff549
        |   f000:f53a      b91b00         mov cx, 0x1b
        |   f000:f53d      0f32           rdmsr  ; check BSP (Boot Strap Processor) flag, if set, loop back to 0xffff0; otherwise, infinite hlt
        |   f000:f53f      f6c401         test ah, 1
       ,==< f000:f542      7441           je 0xff585
      ,===< f000:f544      eaf0ff00f0     ljmp 0xf000:0xfff0
      ||`-> f000:f549      b001           mov al, 1
      ||    f000:f54b      e680           out 0x80, al
      ||    f000:f54d      66be8cfdffff   mov esi, 0xfffffd8c          ; 4294966668
      ||    f000:f553      662e0f0114     lgdt cs:[si]
      ||    f000:f558      0f20c0         mov eax, cr0
      ||    f000:f55b      6683c803       or eax, 3
      ||    f000:f55f      0f22c0         mov cr0, eax
      ||    f000:f562      0f20e0         mov eax, cr4
      ||    f000:f565      660d00060000   or eax, 0x600
      ||    f000:f56b      0f22e0         mov cr4, eax
      ||    f000:f56e      b81800         mov ax, 0x18
      ||    f000:f571      8ed8           mov ds, ax
      ||    f000:f573      8ec0           mov es, ax
      ||    f000:f575      8ee0           mov fs, ax
      ||    f000:f577      8ee8           mov gs, ax
      ||    f000:f579      8ed0           mov ss, ax
      ||    f000:f57b      66be92fdffff   mov esi, 0xfffffd92          ; 4294966674
      ||    f000:f581      662eff2c       ljmp cs:[si]
      |`.-> f000:f585      fa             cli
      | :   f000:f586      f4             hlt
      | `=< f000:f587      ebfc           jmp 0xff585

To conclude the weirdness, this BIOS code is reading itself at 0xffff0 and comparing the byte with 0xea, which is exactly the opcode of a far jump:

            f000:f52a      b800f0         mov ax, 0xf000
            f000:f52d      8ec0           mov es, ax
            f000:f52f      6726a0f0ff00.  mov al, byte es:[0xfff0]     ; [0xfff0:1]=0
            f000:f536      3cea           cmp al, 0xea

If it finds the code at 0xffff0 is a far jump, then it will go into an infinite loop.

More precisely, the APs (Application Processors) will loop infinitely at the hlt instruction, while the BSP (Boot Strap Processor) will loop back to the beginning 0xffff0. Since the code at 0xffff0 won't be changed, we can conclude the BSP will always find the byte being 0xea and will never go out of the loop.

So what's the purpose of this self-checking? I can hardly believe it's a naive attempt to prevent modification.

like image 317
user10865622 Avatar asked Jan 20 '19 06:01

user10865622


2 Answers

Albeit hard to reason about, remember that the load mov al, byte es:[0xfff0] is not reading from the the BIOS first instruction, even though es is set to 0xf000.

The first instruction is read from 0xfffffff0, the PCH will also probably alias 0xf0000-0xfffff to 0xffff0000-0xffffffff at reset, so when the BSP is booted it will execute the code you dumped.
IIRC, the APs don't boot unless explicitly waken up.

The BSP will then will proceed with initialising the HW (judging from the dump).
At some point it will set the attribute map for the 0xf0000-0xfffff to steer reads and writes (or just writes and then reads) to memory.
The end result is that when a processor (an HW thread) boots it will execute the code from the flash until it perform a far jump.
At the point the cs base is correctly computed as per real-mode rules (pretty much like the unreal mode) and the instruction will be fetched from the 0xf0000-0xfffff (i.e. from the RAM).
All of this while the cs segment value didn't actually change.

The BSP at some point will start its multiprocessor initialisation routine, where it broadcasts to everyone (including himself) an INIT-SIPI-SIPI that will result in a sleep for the APs and a ljmp 0xf000:0xfff0 for the BSP.
The trick here is that the target of the jump, 0xf000:0xfff0, is not the same bus address of the wbinvd instruction.
There could be something else there, probably another initialisation routine.

At the end of the initialisation the BIOS could simply reset the attributes of the 0xf0000-0xfffff to fall through to the flash (so a software reset is possible), preventing (not intentionally) a dump of the intermediary code.

This is not very efficient, but BIOSes are not usually masterpieces of code.

I don't have enough element to be sure what's going on, my point is that the ljmp 0xf000:0xfff0 and the mov al, byte es:[0xfff0] doesn't have to read from the same region they reside in.
With this in mind, all bets are off.
Only a proper reverse engineering will tell.

Regarding the wbinvd, I suggested in the comment it could be related to the warm boot facility and Peter Cordes suggested that it may specifically have to do with cache-as-RAM.
It makes sense, I guess will never be sure though.
It could as well be a case of cargo cult, where a programmer deemed the instruction necessary based rumors.

like image 183
Margaret Bloom Avatar answered Oct 17 '22 12:10

Margaret Bloom


This is actually the answer to the title question:

Hadi Brais: According to slide 14 of BIOS and System Management Mode Internals, the wbinv instruction was there in UDK2010 but then got later removed in UDK2012. Perhaps it's security-related. I don't know exact what.

I can confirm that this instruction is not present at 0xfffffff0 on my BIOS version from 2016.

There is a more burning question here and that's what does the comparison with comparison with 0xea mean.

The reset vector at 0xfffffff0 contains 90 90 E9 43 FC, which is a relative jump to 0xfffffff5-3bd which is 0xfffffc38, the entry point of my SEC Core PE32 image from 0xffffca18 - 0xffffffbb:

0x00:  DB E3                      fninit 
0x02:  0F 6E C0                   movd   mm0, eax   //move BIST value to mm0
0x05:  0F 31                      rdtsc  
0x07:  0F 6E EA                   movd   mm5, edx
0x0a:  0F 6E F0                   movd   mm6, eax  //save tsc
0x0d:  66 33 C0                   xor    eax, eax //clear eax

0x10:  8E C0                      mov    es, ax
0x12:  8C C8                      mov    ax, cs
0x14:  8E D8                      mov    ds, ax
0x16:  B8 00 F0                   mov    ax, 0xf000
0x19:  8E C0                      mov    es, ax
0x1b:  67 26 A0 F0 FF 00 00       mov    al, byte ptr es:[0xfff0]
0x22:  3C EA                      cmp    al, 0xea
0x24:  74 0E                      je     0x34   //if ea is at ffff0h then jump to the 0xf000e05b check 

0x26:  BA F9 0C                   mov    dx, 0xcf9
0x29:  EC                         in     al, dx    //read port 0xcf9
0x2a:  3C 04                      cmp    al, 4    
0x2c:  75 25                      jne    0x53      
0x2e:  BA F9 0C                   mov    dx, 0xcf9 //perform hard reset since if CPU only reset is issued not all MSRs are restored to their defaults
0x31:  B0 06                      mov    al, 6
0x33:  EE                         out    dx, al  

0x34:  67 66 26 A1 F1 FF 00 00    mov    eax, dword ptr es:[0xfff1]
0x3c:  66 3D 5B E0 00 F0          cmp    eax, 0xf000e05b
0x42:  75 0F                      jne    0x53      //if the ptr16:16 of the EA instruction isn't 0xf000e05b, move to notwarmstart

0x44:  B9 1B 00                   mov    cx, 0x1b //if it is equal, read bsp bit from apic_base msr
0x47:  0F 32                      rdmsr  
0x49:  F6 C4 01                   test   ah, 1
0x4c:  74 41                      je     0x8f   //if the and operation with 00000001b produces a zero result i.e. it's an AP then jump to cli, hlt

0x4e:  EA F0 FF 00 F0             ljmp   0xf000:0xfff0 //if it's the BSP and legacy bios is present, far jump to 0xffff0, exiting unreal mode

notwarmstart:
0x53:  B0 01                      mov    al, 1
0x55:  E6 80                      out    0x80, al  //send 1 as a debug POST code
0x57:  66 BE 68 FF FF FF          mov    esi, 0xffffff68
0x5d:  66 2E 0F 01 14             lgdt   cs:[si] //loads 32&16 GDT pointer (not 16&6, due to 66 prefix) at 16bit address fff68 in si into GDTR (base:ffffff28 limit:003f).

//enter 16 bit protected mode//
0x62:  0F 20 C0                   mov    eax, cr0
0x65:  66 83 C8 03                or     eax, 3   //Set PE bit (bit #0) & MP bit (bit #1)
0x69:  0F 22 C0                   mov    cr0, eax  //Activate protected mode
0x6c:  0F 20 E0                   mov    eax, cr4 
0x6f:  66 0D 00 06 00 00          or     eax, 0x600 //Set OSFXSR bit (bit #9) & OSXMMEXCPT bit (bit #10)
0x75:  0F 22 E0                   mov    cr4, eax

//set up selectors for 32 bit protected mode entry
0x78:  B8 18 00                   mov    ax, 0x18 //segment descriptor at 0x18 in GDT is (raw): 00cf93000000ffff
0x7b:  8E D8                      mov    ds, ax
0x7d:  8E C0                      mov    es, ax
0x7f:  8E E0                      mov    fs, ax
0x81:  8E E8                      mov    gs, ax
0x83:  8E D0                      mov    ss, ax
0x85:  66 BE 6E FF FF FF          mov    esi, 0xffffff6e
0x8b:  66 2E FF 2C                ljmp   cs:[si]   //transition to flat 32 bit protected mode and jump to address at 0x0:0xffffff6e aka. 0xffffff6e which is fffffcd8. CS contains 0 remember (it's the base that is 0xffff) so it will load the first entry.
                                                   //SEC continues at that address

0x8f:  FA                         cli    
0x90:  F4                         hlt    
.
.

We notice that my code differs from yours. There is an extra comparison to 0xf000e05b and a read/write to 0xcf9. Also, your system contains the e9 jump code we see at 0xfffffff0 at 0xffff0 as well, this is because you must be in UEFI boot and therefore there is no legacy BIOS shadowed to the legacy BIOS range at 0xf0000. Yours contains a wbinvd before the relative jump and mine contains 2 nops.

A clue here in the edk2 source code is that the code being jumped to is called 'NotWarmStart'. The code speaks for itself.

In mine, if EA is at 0xffff0 then it checks 0xffff1 for 0xf000e05b. If 0xf000e05b is there then it checks for the BSP flag, and if its the BSP, it jumps to 0xffff0. If 0xf000e05b isn't there, it jumps to the 16 bit + 32 bit protected mode setup (called 'NotWarmStart), which then jumps to 32 bit flat protected mode (edk2 calls this PEI, but I'd say PEI classically begins at the PEI core and that the code it jumps to is actually still SEC (given that this code uses FSP to set up CAR, optionally perform microcode updates if BootGuard isn't present and then passes control to the PEI core, and it's quite clearly an address in the code section of my SEC Core PE32 image, whereas PEI core image entry is at 0xffe91854)) implementation at 0x18:0xffffff6e. If EA is not present, it checks bit 3 of 0xcf9 for 'Check INIT# is asserted'. If it is asserted then it performs a warm reset, writing 0x6 which results in a PLTRST#, reason 'issue warm start, since if CPU only reset is issued not all MSRs are restored to their defaults'. If it isn't asserted then it jumps to 'NotWarmStart'.

On my system, both 0xffff0 and 0xfffffff0 are being redirected to SPI flash. You can disable the 0xffff0 range with BIOS_Legacy_F_EN, but when enabled, it always goes to where the range containing 0xfffffff0 is being redirected, you can't redirect one to SPI and another to LPC/eSPI. A cold legacy boot goes through SEC and PEI before it is extracted from the SPI flash and shadowed into RAM at 0xe0000 and/or 0xf0000 by redirecting writes to the memory controller in the Cbo SAD PAMs for those ranges but not reads, until the shadow is complete. On my system, the legacy bios is shadowed to 0xf0000.

My current guess is:

  • Containing 0xea and 0xf000e05b means that it is a legacy soft reset (CPU warm reset) or S3 resume and the legacy BIOS has already been shadowed into RAM, so let the legacy BIOS deal with the S3 resume or CPU warm reset.
  • Containing 0xea and not 0xf000e05b.... f000:e05b appears to be the POST entry point. Why it wouldn't contain that I don't 100% know, but of course, either the legacy BIOS or the UEFI bios implementation that shadowed in the legacy BIOS in the first place can change this, and when it's been changed to that it needs to go to SEC.
  • Not containing 0xea means either UEFI/legacy warm reset, UEFI soft reset, UEFI S3 resume or UEFI/legacy cold boot/reset, and if it is a soft reset, downgrade to warm reset, otherwise it needs to go to SEC.

Also there is no loop at hlt. Hlt enters a HALT C1 state, and responds to an INIT# IPI to put it in a wait-for-SIPI state. Execution will then begin at whatever address the BSP selects for the AP.

On my Windows 7 VirtualBox, which doesn't have a UEFI BIOS at all, both the 0xf0000 and 0xffff0000 map to the virtual LPC flash and therefore both show the BIOS.

kd> !db [uc] 00000000`ffffff80
#ffffff80 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#ffffff90 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#ffffffa0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#ffffffb0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#ffffffc0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#ffffffd0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#ffffffe0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 58 4d ..............XM
#fffffff0 ea 5b e0 00 f0 30 36 2f-32 33 2f 39 39 00 fc 8f .[...06/23/99...
kd> !db [uc] fff80
#   fff80 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fff90 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffa0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffb0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffc0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffd0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffe0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 58 4d ..............XM
#   ffff0 ea 5b e0 00 f0 30 36 2f-32 33 2f 39 39 00 fc 8f .[...06/23/99...

On my legacy boot UEFI:

 ffffff80 FF 46 A0 70 CC CC CC CC E9 1B F6 FF FF E9 26 F6  Fáp¦¦¦¦Ú.÷  Ú&÷
 ffffff90 FF FF E9 71 F6 FF FF E9 7C F6 FF FF E9 3C FD FF   Úq÷  Ú|÷  Ú<² 
 ffffffa0 FF E9 3C FD FF FF E9 75 FF FF FF 00 00 00 00 00  Ú<²  Úu   .....
 ffffffb0 00 00 00 00 00 00 00 00 04 00 00 19 44 00 00 19 ............D...
 ffffffc0 00 01 D9 FF 00 00 00 00 00 00 00 00 00 00 00 00 ..+ ............
 ffffffd0 BF 50 41 EB 1D 00 00 00 00 00 00 00 00 00 00 00 +PAÙ............
 ffffffe0 54 18 E9 FF EB FE CF 00 00 00 00 00 00 00 00 00 T.Ú Ù¦¤.........
 fffffff0 90 90 E9 43 FC 00 00 00 FB 00 00 00 00 00 E9 FF ..ÚC³...¹.....Ú 

lkd> !db fff80
#   fff80 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fff90 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffa0 00 00 00 00 00 00 00 00-00 00 e9 e3 3b 00 00 00 ............;...
#   fffb0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffc0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffd0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   fffe0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
#   ffff0 ea 5b e0 00 f0 30 37 2f-32 32 2f 31 36 00 fc 00 .[...07/22/16...
like image 30
Lewis Kelsey Avatar answered Oct 17 '22 10:10

Lewis Kelsey