How to switch from real mode to protected mode after bootloader?

Question

I just finished up a very bare-bones bootloader for my OS and now I'm trying to switch to protected mode and jump to the kernel.

The kernel exists on the second sector (right after the bootloader) and on.

Can anyone help me out with my code? I added comments to show where my confusion is.

Thank you.

BITS 16 

global start
start:
    ; initialize bootloader and stack
    mov     ax, 0x07C0
    add     ax, 288
    mov     ss, ax
    mov     sp, 4096
    mov     ax, 0x07C0
    mov     ds, ax

    call    kernel_load
    hlt

kernel_load:
    mov     si, k_load
    call    print

    mov     ax, 0x7C0
    mov     ds, ax
    mov     ah, 2
    mov     al, 1
    push    word 0x1000
    pop     es
    xor     bx, bx
    mov     cx, 2
    mov     dx, 0
    int     0x13

    jnc     .kjump
    mov     si, k_fail
    call    print
    ret

.kjump:
    mov     si, k_succ
    call    print

    ; this is where my confusion starts

    ; switch to protected mode???
    mov     eax, cr0
    or      eax, 1
    mov     cr0, eax

    ; jump to kernel? 
    jmp     0x1000:0

    hlt

data:
    k_load  db "Initializing Kernel...", 10, 0
    k_succ  db "Kernel loaded successfully!", 10, 0
    k_fail  db "Kernel failed to load!", 10, 0

print:
    mov     ah, 0x0E
.printchar:
    lodsb
    cmp     al, 0
    je      .done
    int     0x10
    jmp     .printchar
.done:
    ret

times 510-($-$$) db 0
dw 0xAA55

Answer 1

You need to setup several things before you attempt to enter protected mode:

Initialize a GDT in memory

You need a global descriptor table in memory. It needs room for at least these selectors:

• You need a ring0 32-bit code descriptor
• You need a ring0 32-bit data descriptor
• You need a GDT segment
• You need an IDT segment
• You need a TSS segment
• You probably want an LDT segment (every process should have an LDT that begins at the same linear address in every process, then the one LDT descriptor can handle every process, and paging will handle the switching).

In protected mode, a selector is an index into the GDT or LDT. Code and data descriptors tell the CPU the base address and length of the memory to use when a selector is loaded with that index.

The LGDT instruction sets the GDTR.

Initialize a TSS in memory

A TSS segment tells the CPU where you are going to store the TSS. Some of the functionally originally built into the TSS is marginally useful, since the context switch is faster if you do it manually. However, it is essential for one thing though: it stores the stack for the kernel to use when a process transitions from ring3 to ring0. The kernel cannot trust the caller at all. It cannot assume that the caller did not go crazy and corrupt the stack pointer. When transitioning from ring3 to ring0, the CPU loads the stack pointer from the TSS, and pushes the callers stack segment and offset onto the kernel stack, before pushing the code segment and offset return address.

The LTR instruction loads the task register with a TSS segment.

Initialize an IDT in memory

The IDT lets the CPU lookup what to do when various events occur. The essential purpose is exception handling. The CPU implements exceptions as interrupts. The operating system must set up handlers for all of the exceptions.

The LIDT instruction loads the IDTR.

Hardware interrupts covered below.

If an exception occurs while processing an exception, a double fault exception occurs. If an exception occurs when processing a double fault, the CPU translates that into a shut-down message to the motherboard. Typical motherboards will reset the CPU when that happens, the BIOS will see that the reset was unexpected in its bootstrap start-up code and it will do a reboot.

Initialize the interrupt controller

Hardware devices also provide hardware interrupts (as opposed to the software interrupts mentioned earlier). Hardware interrupts occur when devices need service.

If you intend to support old machines, then you need code to use and handle the 8259 interrupt controller.

You need code to handle the interrupt, save the context, acknowledge the interrupt, and somehow call a driver or queue a work item somewhere to service the hardware.

The interrupt controller is set up to provoke the CPU to process an interrupt, when a hardware device asserts its interrupt control line (on ancient systems), or when a MSI interrupt packet reaches the CPU (on modern systems capable and configured to use MSI).

If you want maximum capabilities and need to support multiple processors, then you must...

Initialize the APIC

The APIC is exactly what the name says: Advanced Programmable Interrupt Controller.

The APIC allows complex control over prioritization, masking, and interprocessor communication. It is too large and complex to really cover it properly here.

Initialize paging

The paging is broken down into a two level lookup. The top level is called the page directory. The second level is called a page table.

Every page consists of 1024 32 bit page descriptors. The high 20 bits are the high 20 bits of the physical address for that page table entry. Lower bits contain several flags for permission and to let the OS detect usage of memory so it can be intelligently swapped/discarded/kept.

Each page directory entry describes the base address of one 4KB page table for that range of memory. Each entry of the page directory points to one page table which can have up to 4MB of memory mapped.

Each page descriptor of the page table describes the permissions to, access history, and base address of a 4KB range of memory.

So the operating system must allocate at least one 4KB page for the page directory, and at least one 4KB page for every 4MB of memory committed. Note that you may have sparse mappings where there are large regions where no memory exists and a page fault would occur if you accessed it.

You enable paging with the PG bit of CR0. The PDBR control register (CR3) tells the CPU the physical address of the page directory.

Order

Initialize GDT, IDT, TSS (and allocate kernel stack memory, user stack memory (if needed), in memory.

Whack a GDT code and data entry at index 1 and 2 of the GDT memory, and set them to have zero base address, 4GB limit, ring0.

Set CR0 bit 0, the PE or protection-enable bit.

The big jump

Immediately do a far jump to 0x10:next-instruction where next-instruction is probably resolved in the linker to a label on the next line. (You can push can a far pointer on the stack and far jump indirect through it). You need to subtract (cs << 4) from the base address because the jump target is relative to the segment you are assembling at some arbitrary base, set in the real-mode cs.

You must load all of the segment registers after entering protected mode, because the CPU does a bunch of permission checks and sets up several internal things in the CPU that are different in protected mode.

Tell the assembler!

Note that after that branch target, you suddenly need to start assembling instructions differently. Before the far jump, you were in real mode, but as soon as cs loaded, a whole lot of things changed in the CPU, and it actually changes the way it decodes instructions. It assumes 32-bit registers and addresses, and the address size prefix tells it to be 16-bit.

In real mode, it was the other way around, the address size or operand size prefix told it to be 32-bit. Therefore you need to use some kind of assembler directive to tell the assembler to reverse the usage of those prefixes and change various things to deal with 32-bit mode.

Obviously you need to setup the stack. You had to deal with linear addresses several times already, when setting up the descriptor addresses for LDT,IDT, etc.

Now you can setup the page directory and page tables, load the PBDR.

Each page directory entry can be flagged to not be flushed when switching page tables. Typically kernel mode has the same mapping for every process.

Typically each process gets its own page directory, and it shares the kernel tables. Its user mode allocations are done to its own private page tables for the user memory range.

Although paging is not required, it enables a lot of really cool capabilities and protections. You probably want it.

After you enable paging and load the PDBR, you are, by every definition, completely in protected mode, and you have implemented a chunk of the core code to implement an operating system on the x86 architecture.

Answer 2

@doug65536's answer is very extensive and expressive, but excess. It covers all necessary to bring the processor into a final working state required by most operating systems. However, when we talk about switching from the Real 16-bit into Protected 32-bit mode exclusively on x86 architecture, we need to perform much fewer actions that are indeed necessary to accomplish the switch.

1. Your code needs to know the layout of physical memory.

The reason is that your code should touch data in memory to switch into protected mode. Thus, you need to know where this data locate. The sad part of the story is that the only way to deliver such crucial addresses into your code is to use instructions with immediate arguments (like mov ax, 07C00h). Thus, you are enforced to know where your code and data will be placed in memory at compilation time!

2. You should properly set up the Data Segment register.

The reason is the same: you will need touch data in memory, so you should address this data correctly. In x86 Real Mode processor employs a segmentation model. In this model, the CPU constructs a 20-bit memory address from two parts: implicitly used 16-bit segment address and explicitly specified 16-bit pointer (segment * 16 + ptr). Dedicated 16-bit segment registers store segment addresses. You generally need two of them: Code Segment register (CS) and Data Segment register (DS). The good news is that if your code is already executing, then CS is already initialized. So, you do not need to care about it. The bad news is that the same story does not work with DS, and it is your responsibility to set it up before touching memory.

3. You need to have or build Global Descriptor Table (GDT) in memory.

GDT is a central data structure managing the processor in Protected Mode. Generally, you need at least three segments:

• NULL-segment descriptor. It is a mandatory first item of GDT on x86.
• Code Segment. It is required to switch to Protected Mode.
• Data Segment. This item is not needed directly to perform the switch. However, once you are already in protected mode, you will need to access some data in memory for sure. So, you have to have a Data Segment descriptor in GDT if you want to do something useful after the switch.

4. You need to have or build Global Descriptor Table FWORD-pointer in memory.

To load GDT into the CPU, you need to have an in-memory FWORD descriptor of GDT. This descriptor consists of a 16-bit size of GDT in bytes minus one and a 32-bit linear address of GDT in memory. And yes, this descriptor must be located in memory too.

5. You need to load GDT.

Once you have 1-4 done (you have GDT and its FWORD descriptor, you know their addresses, and you have DS initialized to address them), you can and should load GDT into CPU using lgdt instruction. After that, not only you personally, but your processor too, will be aware of the GDT.

6. You should disable interrupts before switch.

After the switch, the CPU changes the strategy of interrupt handling. The one used in Real Mode and based on dispatch through Interrupt Vector Table (IVT) and set by BIOS will not work anymore. To handle interrupt in Protected Mode, you need to set up and load Interrupt Descriptor Table (IDT). Once you come to PM and until you have IDT loaded, any interrupt arrived at the CPU will crash your system. Therefore, BEFORE you switch, you must disable interrupts using the cli instruction or have IDT prepared and loaded.

7. Enable PM mode in CR0.

After 1-6 accomplished (GDT loaded and interrupts disabled), you can finally enable Protected Mode by raising the appropriate flag in the CR0 register.

8. Jump into Protected Mode.

Last but not least, immediately after enabling PM in CR0 (mov CR0, reg) instruction, you should place a jmp dword imm16:imm32 one, where 'imm16' is replaced by the selector of Code Segment from your GDT and imm32 is replaced by target address of the jump. This instruction reloads the memory segmentation mechanics of the CPU (set CS register) and allows the CPU to continue to fetch and execute instructions. Without this step, the protection subsystem of memory segmentation will crash your code almost immediately. Note! You are not allowed to place any additional instructions between mov CR0, reg and jmp dword imm16:imm32 if your goal is not a system crash!

---

That's it!

The only data structure you need to switch from Real Mode to Protected Mode is GDT correctly set and loaded.

Everything else (preparing IDT, loading TSS, reloading new GDT, etc.) is not strictly necessary to switch into Protected Mode. And you can accomplish these tasks when you are already in Protected Mode.