Qemu-KVM: Translation of guest physical address to host virtual/host physical address

Question

I am working on a project where I need to translate qemu-guest physical addresses to host virtual/physical addresses.

I am using VMI (virtual machine introspection) to introspect into the qemu process (the KVM VM) and to read guest physical addresses stored in virtio ring buffer descriptors. Therefore, I am looking for a simple way to translate the qemu physical addresses to host virtual addresses at the host side. (i.e., to extract as less info as possible from the qemu process).

I read online that in previous versions, qemu stored the physical RAM base in the variable phys_ram_base, so that the host virtual address could be obtained as follows:

host_virtual = phys_ram_base + guest_physical_address

Is something like this possible in newer versions of qemu (e.g., how could I obtain the qemu-physical base address -- the former phys_ram_base?)

Answer 1

I had to solve the same problem: translating a guest-virtual address to a host-physical address. My approach was the following:

Step 1 (native host): virtual address to physical address

• I wrote a program vaddr2paddr that takes a process ID (PID) and a virtual address, then returns the associated physical address.
• This program uses /proc/<pid>/pagemap to determine the physical address of the program
• I derived the code from the blogpost Translating Virtual Addresses to Physical Addresses in User Space and the dwks/pagemap tool.
• I tested this script thoroughly on my native host with a simple test program that allocates a buffer and prints the virtual address, then waits for an input and in the meanwhile I run my vaddr2paddr script and then check using devmem2 tool (or with xxd in /dev/mem by loading devmem-full-access kernel module) whether my previously written data is indeed there

Step 2 (VM): guest-virtual address to host-physical address

• I run the same test program that allocates a buffer and writes some data into it in my VM
• I execute vaddr2paddr with the guest-virtual address of the allocated buffer to get the guest-physical address (gpa)
• On the native host, I determine the largest memory region of the qemu process by parsing /dev/<pid>/maps which shows the allocated virtual memory regions of a process. In my case, the VM had 2 GB of memory and I found an area that was roughly 2 GB (all others were significantly smaller). I then take the start address of that area (vm_start_address)
  • For that, I have written a simple Python script but there is also existing code on the web, e.g., ouadev/proc_maps_parser
• Now we can compute the host-physical address of our allocated buffer by: hpa = vm_start_address + gpa
• Again, I verify using devmem2 tool whether the written data is at the calculated hpa

NOTE: This approach requires sudo to access /proc/<pid>/pagemap and also /proc/<pid>/maps.

Answer 2

I had to solve the same problem and I come up with the following solution.

When using QEMU with the -enable-kvm option, memory is allocated to the guest through the KVM_SET_USER_MEMORY_REGION ioctl. Basically, QEMU prepares a kvm_userspace_memory_regionstruct, where the physical addresses of the guest are associated to host virtual addresses, and then the ioctl is issued. Now, it turned out that the KVMSlot struct is (almost) 1:1 with the struct offered by the KVM API. QEMU stores all the information to perform the translation from guest physical to host virtual addresses there.

The KVMSlot struct is defined like this:

    typedef struct KVMSlot
    {
        hwaddr start_addr;
        ram_addr_t memory_size;
        void *ram;
        int slot;
        int flags;
        int old_flags;
        /* Dirty bitmap cache for the slot */
        unsigned long *dirty_bmap;
    } KVMSlot;

start_addr is the physical address corresponding to the begininnig of the considered slot, ram is its corresponding host virtual address and then memory_size is the size of the slot.

Now to perform the translation you have to:

1. Find the right slot. Many slots can be allocated and they are kept in a list of KVMSlot elements. The head of the list is stored in KVMMemoryListener. To find it, you can check if the guest physical address is in the range between start_addr and start_addr + memory_size.
2. Compute the offset of your guest physical address in the slot (offset = gpa - start_addr)
3. Compute the translated host virtual address like hva = ram + offset. The offset of course is the same both in the guest physical addresses and in the host virtual addresses, that's why you can use it.

Finally you can check that the translation was right using the function gpa2hva of the QEMU Monitor.