How to set the maximum priority to a Slurm job?

Question

as administrator I need to give the maximum priority to a given job.

I have found that submission options like: --priority=<value> or --nice[=adjustment] could be useful, but I do not know which values I should assign them in order to provide the job with the highest priority.

Another approach could be to set a low priority by default to all the jobs and to the special ones increase it.

Any idea of how I could carry it out?

EDIT: I am using sched/backfill policy and the default job priority policy (FIFO).

Thank you.

Answer 1

I found a solution that works without the need of using PriorityType=priority/multifactor (as suggested by Bub Espinja):

$ scontrol update job=<job-id> Priority=<any-integer>

The above command will update the priority of the job and update the queue accordingly.

The minimum priority needed to become the next one in line can be found by checking the priority of the next pending job and adding one to it. You can find the priority of a job using the following:

$ scontrol show job=<job-id>

(scontrol update can be used to change many aspects of a job, such as time limit and others.)

EDIT:

I just learned one can do

$ scontrol top <job-id>

to put a job at the top of their queue.

Answer 2

What I have done is to use the priority plug-in multifactor, with the default configuration, adding this line to slurm.conf:

PriorityType=priority/multifactor

Then, as all the jobs will have the priority 0, I must update the target job priority, in my case using the API:

job_desc_msg_t job_update;
slurm_init_job_desc_msg(&job_update);
job_update.job_id = target_job_id;
job_update.priority = 4294967295;
slurm_update_job(&job_update);

EDITED:

From the Slurm FAQ:

The job's priority is an integer that ranges between 0 and 4294967295. The larger the number, the higher the job will be positioned in the queue, and the sooner the job will be scheduled.