Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

SLURM job history: get full length JobName

Tags:

slurm

I want to get information about my job history of SLURM jobs. I use something like

sacct --starttime 2014-07-01 --format=User,JobID,Jobname,partition,state,time,start,end,elapsed,MaxRss,MaxVMSize,nnodes,ncpus,nodelist

to get a summary of my jobs, but it is difficult to keep track with the JobName section only showing a small part of my job names. I have many jobs where the name shares several words. I would the sacct command to show more of the job name, and preferentially the whole name. I notice that other column widths are made to fit the information below, so why not for JobName?

like image 567
Yoda Avatar asked Jan 10 '18 12:01

Yoda


People also ask

How do I get the status of a SLURM job?

Information on all running and pending batch jobs managed by SLURM can be obtained from the SLURM command squeue. Note that information on completed jobs is only retained for a limited period. Information on jobs that ran in the past is via. sacct An example of the output squeue is shown below.

What is Slurm and how does it work?

Slurm also provides a utility to hold jobs that are queued in the system. Holding a job will place the job in the lowest priority, effectively “holding” the job from being run. A job can only be held if it’s waiting on the system to be run.

What is a job ID in Slurm?

When you first submit your job, SLURM should give you a job ID which represents the resources allocated to your job. Individual calls to srun will spawn job steps which can also be queried individually.

How do I view job accounting information in Slurm?

Accounting information for jobs invoked with Slurm are either logged in the job accounting log file or saved to the Slurm database, as configured with the AccountingStorageType parameter. The sacct command displays job accounting data stored in the job accounting log file or Slurm database in a variety of forms for your analysis.


1 Answers

From the sacct man page:

   -o, --format
             Comma separated list of fields. (use "--helpformat" for a list of available fields).
             NOTE: When using the format option for listing various fields you can put a %NUMBER afterwards to specify how many characters should be printed.
             e.g. format=name%30 will print 30 characters of field name right justified.  A %-30 will print 30 characters left justified.
             When set, the SACCT_FORMAT environment variable will override the default format.  For example:
             SACCT_FORMAT="jobid,user,account,cluster"

So you can simply specify the length of the Jobname field by adding %NUMBER after it.

It will look something like this:

sacct --starttime 2014-07-01 --format=User,JobID,Jobname%50,partition,state,time,start,end,elapsed,MaxRss,MaxVMSize,nnodes,ncpus,nodelist

PS: try to avoid querying too many values from the database as it may take a long time and affect the correct behaviour of slurm. So reduce the time range to a reasonable value.

like image 149
Carles Fenoy Avatar answered Nov 15 '22 11:11

Carles Fenoy