Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to use sched_getaffinity and sched_setaffinity in Linux from C?

I am trying to:

  • Run 16 copies concurrently with processor pinning (2 copies per core)

  • Run 8 copies concurrently with processor pinning (2 copies per core) and flipping processor core to the furthest core after certain function say function 1 finishes.

The problem I am facing is how to select the farthest processor.

Some friends suggested to use sched_getaffinity and sched_setaffinity but I count not find any good examples.

like image 598
Basmah Avatar asked May 07 '12 23:05

Basmah


People also ask

What is Sched_setaffinity?

sched_setaffinity() sets the CPU affinity mask of the thread whose ID is pid to the value specified by mask. If pid is zero, then the calling thread is used. The argument cpusetsize is the length (in bytes) of the data pointed to by mask. Normally this argument would be specified as sizeof(cpu_set_t).

How do I check my CPU affinity?

CPU affinity enables binding a process or multiple processes to a specific CPU core in a way that the process(es) will run from that specific core only. When trying to perform performance testing on a host with many cores, it is wise to run multiple instances of a process, each one on different core.

What is CPU affinity Linux?

As per Wikipedia, Processor affinity, or CPU pinning or “cache affinity”, enables the binding and unbinding of a process or a thread to a central processing unit (CPU) or a range of CPUs, so that the process or thread will execute only on the designated CPU or CPUs rather than any CPU.

What is thread affinity in Linux?

Thread affinity provides a way for an application thread to tell the OS scheduler exactly where its threads can (and would like to) run. The scheduler in turn does not have to spend a lot of time load balancing the system because application threads are already where they need to be.


2 Answers

To use sched_setaffinity to make the current process run on core 7 you do this:

cpu_set_t my_set;        /* Define your cpu_set bit mask. */
CPU_ZERO(&my_set);       /* Initialize it all to 0, i.e. no CPUs selected. */
CPU_SET(7, &my_set);     /* set the bit that represents core 7. */
sched_setaffinity(0, sizeof(cpu_set_t), &my_set); /* Set affinity of tihs process to */
                                                  /* the defined mask, i.e. only 7. */

See http://linux.die.net/man/2/sched_setaffinity & http://www.gnu.org/software/libc/manual/html_node/CPU-Affinity.html for more info.

like image 73
Pete Fordham Avatar answered Sep 27 '22 23:09

Pete Fordham


Minimal runnable example

In this example, we get the affinity, modify it, and check if it has taken effect with sched_getcpu().

main.c

#define _GNU_SOURCE
#include <assert.h>
#include <sched.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

void print_affinity() {
    cpu_set_t mask;
    long nproc, i;

    if (sched_getaffinity(0, sizeof(cpu_set_t), &mask) == -1) {
        perror("sched_getaffinity");
        assert(false);
    }
    nproc = sysconf(_SC_NPROCESSORS_ONLN);
    printf("sched_getaffinity = ");
    for (i = 0; i < nproc; i++) {
        printf("%d ", CPU_ISSET(i, &mask));
    }
    printf("\n");
}

int main(void) {
    cpu_set_t mask;

    print_affinity();
    printf("sched_getcpu = %d\n", sched_getcpu());
    CPU_ZERO(&mask);
    CPU_SET(0, &mask);
    if (sched_setaffinity(0, sizeof(cpu_set_t), &mask) == -1) {
        perror("sched_setaffinity");
        assert(false);
    }
    print_affinity();
    /* TODO is it guaranteed to have taken effect already? Always worked on my tests. */
    printf("sched_getcpu = %d\n", sched_getcpu());
    return EXIT_SUCCESS;
}

GitHub upstream.

Compile and run:

gcc -ggdb3 -O0 -std=c99 -Wall -Wextra -pedantic -o main.out main.c
./main.out

Sample output:

sched_getaffinity = 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
sched_getcpu = 9
sched_getaffinity = 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
sched_getcpu = 0

Which means that:

  • initially, all of my 16 cores were enabled, and the process was randomly running on core 9 (the 10th one)
  • after we set the affinity to only the first core, the process was moved necessarily to core 0 (the first one)

It is also fun to run this program through taskset:

taskset -c 1,3 ./a.out

Which gives output of form:

sched_getaffinity = 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 
sched_getcpu = 2
sched_getaffinity = 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
sched_getcpu = 0

and so we see that it limited the affinity from the start.

This works because the affinity is inherited by child processes, which taskset is forking: How to prevent inheriting CPU affinity by child forked process?

nproc respects sched_getaffinity by default as shown at: How to find out the number of CPUs using python

Python: os.sched_getaffinity and os.sched_setaffinity

See: How to find out the number of CPUs using python

Tested in Ubuntu 16.04.