/proc/[pid]/pagemaps and /proc/[pid]/maps | linux

Question

I'm trying to get my head around the two files mentioned in the title. I've looked up what the bits are; however, I'm failing to understand how to extract useful info from them (or I'm simply approaching it the wrong way).

Let me explain: The pagemaps is a rather newer "feature" pseudo file that contains the physical frame information of virtual pages assigned to a current [pid]. That is, given a virtual page that starts at address x, say 'vas' for virtual address start, i can index the pagemap file using vas to get the 64bits of the mapped physical page frame. These bits contain info about that virtual page. However, when I extract the bits and do a bit of shifting I'm getting lost with what I'm seeing.

The bits are represented as follows: 0-54 is the page frame number, 55-60 is the page shift, 63rd bit is the present bit, there are other bits of little interest to me. After I do a bit of mapping using vas addresses from /proc/[pid]/maps, it seems that just about every process' page is swapped, i.e. the 63rd bit is always a zero. :(

I guess the question would be, how should I go about effectively using pagemaps to get the equivalent physical address of the address given by /proc/[pid]/maps

To be fair, I've posted a similar question but the approach was a bit different a few days earlier.

If anyone can shed some light on this matter I would be greatly appreciative.

===EDIT===

To address the comment below: I'm reading a line from /proc/[pid]/maps and the lines look like:

00400000-00401000 r-xp 00000000 08:01 8915461 /home/janjust/my_programs/shared_mem 7ffffef1b000-7ffffef3c000 rw-p 00000000 00:00 0 [stack]

Then I'm extracting the number of virtual pages it touches and indexing a binary file /proc/[pid]/pagemaps , and for each virtual page I can extract the physical page it is assigned to.

The output looks like:

00400000-00401000 r-xp 00000000 08:01 8915461 /home/janjust/my_programs/shared_mem num_pages: 1 : 86000000001464C6

One physical address for each virtual page in the virtual range.

The code for reading the line and extracting the physical address is:

74     /* process /proc/pid/maps, by line*/
75     while(fgets(line, 256, in_map) != NULL){
76         unsigned long vas;
77         unsigned long vae;
78         int num_pages;
79 
80         //print line
81         printf("%s", line);
82 
83         /*scan for the virtual addresses*/
84         n = sscanf(line, "%lX-%lX", &vas, &vae);
85         if(n != 2){
86             printf("Involid line read from %s\n",maps);
87             continue;
88         }
89 
90         num_pages = (vae - vas) / PAGE_SIZE;
91         printf("num_pages: %d\n", num_pages);
92 
93         if(num_pages > 0){
94             long index  = (vas / PAGE_SIZE) * sizeof(unsigned long long);
95             off64_t o;
96             ssize_t t;
97 
98             /* seek to index in pagemaps */
99             o = lseek64(pm, index, SEEK_SET);
100             if (o != index){
101                 printf("Error seeking to o:%ld, index:%ld.\n", o, index);
102             }
103 
104             /* map the virtual to physical page */
105             while(num_pages > 0){
106                 unsigned long long pa;
107 
108                 /* Read a 64-bit word from each pagemap file... */
109                 t = read(pm, &pa, sizeof(unsigned long long));
110                 if(t < 0){
111                     printf("Error reading file \"%s\" \n", page_map);
112                     goto next_line;
113                 }
114                 printf(": %016llX\n", pa);

However, although I think I'm getting the right output, the index seems to be either a type mismatch or something else is going on: The output say for instance for the [shared mem] line in maps gives a wrong index; yet I'm still able to scan through the binary file and get the physical page address.

The example of that output is below:

969 7f7f08d58000-7f7f08d59000 rw-s 00000000 00:04 0    /SYSV00003039 (deleted)
970 num_pages: 1
971 Error seeking to o:-1081840960, index:273796065984.
972 : 8600000000148267

Ok, now, lastly I should say that this is under a 64bit OS, and this problem doesn't persist in a 32bit OS.

Answer 1

F**/proc/<pid>/pagemap + /proc/<pid>/maps dump example program**

Here is a pagemap example that converts virtual addresses to physical: Is there any API for determining the physical address from virtual address in Linux?

The following program uses both /proc/<pid>/pagemap + /proc/<pid>/maps to dump page table information to show how they can be used together. Usage:

sudo ./pagemap_dump.out <pid>

Sample output:

addr pfn soft-dirty file/shared swapped present library
400000 12845d 0 1 0 1 /bin/bash
401000 12845e 0 1 0 1 /bin/bash
402000 12845f 0 1 0 1 /bin/bash

This tells us for example that the virtual address 0x400000 maps to the physical address 0x12845d000.

Why sudo is required: https://unix.stackexchange.com/questions/345915/how-to-change-permission-of-proc-self-pagemap-file/383838#383838

This program works in two steps:

• parse the human readable lines lines from /proc/<pid>/maps. This files contains lines of form:

  7ffff7b6d000-7ffff7bdd000 r-xp 00000000 fe:00 658                        /lib/libuClibc-1.0.22.so
  

  which gives us:

  • 7f8af99f8000-7f8af99ff000: a virtual address range that belong to the process, possibly containing multiple pages.
  • /lib/libuClibc-1.0.22.so the name of the library that owns that memory.

• loop over each page of each address range, and ask /proc/<pid>/pagemap for more information about that page, including the physical address.

pagemap_dump.c

#define _XOPEN_SOURCE 700
#include <errno.h>
#include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>

typedef struct {
    uint64_t pfn : 55;
    unsigned int soft_dirty : 1;
    unsigned int file_page : 1;
    unsigned int swapped : 1;
    unsigned int present : 1;
} PagemapEntry;

/* Parse the pagemap entry for the given virtual address.
 *
 * @param[out] entry      the parsed entry
 * @param[in]  pagemap_fd file descriptor to an open /proc/pid/pagemap file
 * @param[in]  vaddr      virtual address to get entry for
 * @return 0 for success, 1 for failure
 */
int pagemap_get_entry(PagemapEntry *entry, int pagemap_fd, uintptr_t vaddr)
{
    size_t nread;
    ssize_t ret;
    uint64_t data;

    nread = 0;
    while (nread < sizeof(data)) {
        ret = pread(pagemap_fd, ((uint8_t*)&data) + nread, sizeof(data) - nread,
                (vaddr / sysconf(_SC_PAGE_SIZE)) * sizeof(data) + nread);
        nread += ret;
        if (ret <= 0) {
            return 1;
        }
    }
    entry->pfn = data & (((uint64_t)1 << 55) - 1);
    entry->soft_dirty = (data >> 55) & 1;
    entry->file_page = (data >> 61) & 1;
    entry->swapped = (data >> 62) & 1;
    entry->present = (data >> 63) & 1;
    return 0;
}

/* Convert the given virtual address to physical using /proc/PID/pagemap.
 *
 * @param[out] paddr physical address
 * @param[in]  pid   process to convert for
 * @param[in] vaddr virtual address to get entry for
 * @return 0 for success, 1 for failure
 */
int virt_to_phys_user(uintptr_t *paddr, pid_t pid, uintptr_t vaddr)
{
    char pagemap_file[BUFSIZ];
    int pagemap_fd;

    snprintf(pagemap_file, sizeof(pagemap_file), "/proc/%ju/pagemap", (uintmax_t)pid);
    pagemap_fd = open(pagemap_file, O_RDONLY);
    if (pagemap_fd < 0) {
        return 1;
    }
    PagemapEntry entry;
    if (pagemap_get_entry(&entry, pagemap_fd, vaddr)) {
        return 1;
    }
    close(pagemap_fd);
    *paddr = (entry.pfn * sysconf(_SC_PAGE_SIZE)) + (vaddr % sysconf(_SC_PAGE_SIZE));
    return 0;
}

int main(int argc, char **argv)
{
    char buffer[BUFSIZ];
    char maps_file[BUFSIZ];
    char pagemap_file[BUFSIZ];
    int maps_fd;
    int offset = 0;
    int pagemap_fd;
    pid_t pid;

    if (argc < 2) {
        printf("Usage: %s pid\n", argv[0]);
        return EXIT_FAILURE;
    }
    pid = strtoull(argv[1], NULL, 0);
    snprintf(maps_file, sizeof(maps_file), "/proc/%ju/maps", (uintmax_t)pid);
    snprintf(pagemap_file, sizeof(pagemap_file), "/proc/%ju/pagemap", (uintmax_t)pid);
    maps_fd = open(maps_file, O_RDONLY);
    if (maps_fd < 0) {
        perror("open maps");
        return EXIT_FAILURE;
    }
    pagemap_fd = open(pagemap_file, O_RDONLY);
    if (pagemap_fd < 0) {
        perror("open pagemap");
        return EXIT_FAILURE;
    }
    printf("addr pfn soft-dirty file/shared swapped present library\n");
    for (;;) {
        ssize_t length = read(maps_fd, buffer + offset, sizeof buffer - offset);
        if (length <= 0) break;
        length += offset;
        for (size_t i = offset; i < (size_t)length; i++) {
            uintptr_t low = 0, high = 0;
            if (buffer[i] == '\n' && i) {
                const char *lib_name;
                size_t y;
                /* Parse a line from maps. Each line contains a range that contains many pages. */
                {
                    size_t x = i - 1;
                    while (x && buffer[x] != '\n') x--;
                    if (buffer[x] == '\n') x++;
                    while (buffer[x] != '-' && x < sizeof buffer) {
                        char c = buffer[x++];
                        low *= 16;
                        if (c >= '0' && c <= '9') {
                            low += c - '0';
                        } else if (c >= 'a' && c <= 'f') {
                            low += c - 'a' + 10;
                        } else {
                            break;
                        }
                    }
                    while (buffer[x] != '-' && x < sizeof buffer) x++;
                    if (buffer[x] == '-') x++;
                    while (buffer[x] != ' ' && x < sizeof buffer) {
                        char c = buffer[x++];
                        high *= 16;
                        if (c >= '0' && c <= '9') {
                            high += c - '0';
                        } else if (c >= 'a' && c <= 'f') {
                            high += c - 'a' + 10;
                        } else {
                            break;
                        }
                    }
                    lib_name = 0;
                    for (int field = 0; field < 4; field++) {
                        x++;
                        while(buffer[x] != ' ' && x < sizeof buffer) x++;
                    }
                    while (buffer[x] == ' ' && x < sizeof buffer) x++;
                    y = x;
                    while (buffer[y] != '\n' && y < sizeof buffer) y++;
                    buffer[y] = 0;
                    lib_name = buffer + x;
                }
                /* Get info about all pages in this page range with pagemap. */
                {
                    PagemapEntry entry;
                    for (uintptr_t addr = low; addr < high; addr += sysconf(_SC_PAGE_SIZE)) {
                        /* TODO always fails for the last page (vsyscall), why? pread returns 0. */
                        if (!pagemap_get_entry(&entry, pagemap_fd, addr)) {
                            printf("%jx %jx %u %u %u %u %s\n",
                                (uintmax_t)addr,
                                (uintmax_t)entry.pfn,
                                entry.soft_dirty,
                                entry.file_page,
                                entry.swapped,
                                entry.present,
                                lib_name
                            );
                        }
                    }
                }
                buffer[y] = '\n';
            }
        }
    }
    close(maps_fd);
    close(pagemap_fd);
    return EXIT_SUCCESS;
}

Answer 2

Oooh K, the index was correct but comparing off64_t o (8bytes) with long index was interpreting o wrong hence why I was getting that error. Ha! this was a stupid mistake. So adding the appropriate header took care of that.

Missing header :-/ sigh fixes the issue of comparing off64_t with a unsigned long.