qstat and long job names

Question

How can I get qstat to give me full job names?

I know qstat -r gives detailed information about the task, but it's too much and the resource requirements are included.

The qstat -r output is like:

 131806 0.25001 tumor_foca ajalali      qw    09/29/2014 15:49:41                                    1 2-100:1
       Full jobname:     tumor_focality-TCGA-THCA-ratboost_linear_svc
       Hard Resources:   distribution=wheezy (0.000000)
                         h_rt=72000 (0.000000)
                         mem_free=15G (0.000000)
                         h_vmem=15G (0.000000)
                         h_stack=256M (0.000000)
       Soft Resources:   
 131807 0.25001 vital_stat ajalali      qw    09/29/2014 15:49:41                                    1 2-100:1
       Full jobname:     vital_status-TCGA-LGG-ratboost_linear_svc
       Hard Resources:   distribution=wheezy (0.000000)
                         h_rt=72000 (0.000000)
                         mem_free=15G (0.000000)
                         h_vmem=15G (0.000000)
                         h_stack=256M (0.000000)
       Soft Resources:   

Right now my only option is to grep the output as I need:

$ qstat -r | grep "Full jobname" -B1
--
 131806 0.25001 tumor_foca ajalali      qw    09/29/2014 15:49:41                                    1 2-100:1
       Full jobname:     tumor_focality-TCGA-THCA-ratboost_linear_svc
--
 131807 0.25001 vital_stat ajalali      qw    09/29/2014 15:49:41                                    1 2-100:1
       Full jobname:     vital_status-TCGA-LGG-ratboost_linear_svc

Can I do it better to have a nicer output?

Answer 1

This on is a bit messy, but it works as a simple solution to have in the command history. All standard tools. Output is pretty much the same as what you get from a normal qstat call, but you won't get the headers:

One-liner:

qstat -xml | tr '\n' ' ' | sed 's#<job_list[^>]*>#\n#g' \
  | sed 's#<[^>]*>##g' | grep " " | column -t

Description of commands:

List jobs as XML:

qstat -xml

Remove all newlines:

tr '\n' ' '

Add newline before each job entry in the list:

sed 's#<job_list[^>]*>#\n#g'

Remove all XML stuff:

sed 's#<[^>]*>##g'

Hack to add newline at the end:

grep " "

Columnize:

column -t

Example output

351996  0.50502  ProjectA_XXXXXXXXX_XXXX_XXXXXX                user123  r   2015-06-25T15:38:41  [email protected]  1
351997  0.50502  ProjectA_XXX_XXXX_XXX                         user123  r   2015-06-25T15:39:26  [email protected]  1
351998  0.50502  ProjectA_XXXXXXXXXXXXX_XXXX_XXXX              user123  r   2015-06-25T15:40:26  [email protected]  1
351999  0.50502  ProjectA_XXXXXXXXXXXXXXXXX_XXXX_XXXX          user123  r   2015-06-25T15:42:11  [email protected]  1
352001  0.50502  ProjectA_XXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXX    user123  r   2015-06-25T15:42:11  [email protected]  1
352008  0.50501  runXXXX69                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352009  0.50501  runXXXX70                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352010  0.50501  runXXXX71                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352011  0.50501  runXXXX72                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352012  0.50501  runXXXX73                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352013  0.50501  runXXXX74                                     usr1     r   2015-06-25T15:49:04  [email protected]  1

Answer 2

Maybe an easier solution: set SGE_LONG_JOB_NAMES to -1, and qstat will figure out the size of the name column:

export SGE_LONG_JOB_NAMES=-1
qstat -u username

Works for me.

Cheers!

Answer 3

This script works pretty well. It looks like it is from cambridge. http://www.hep.ph.ic.ac.uk/~dbauer/grid/myqstat.py

For Python 3:

#!/usr/bin/python
import xml.dom.minidom
import os
import sys
import string    

f=os.popen('qstat -u \* -xml -r')

dom=xml.dom.minidom.parse(f)


jobs=dom.getElementsByTagName('job_info')
run=jobs[0]

runjobs=run.getElementsByTagName('job_list')


def fakeqstat(joblist):
    for r in joblist:
        try:
            jobname=r.getElementsByTagName('JB_name')[0].childNodes[0].data
            jobown=r.getElementsByTagName('JB_owner')[0].childNodes[0].data
            jobstate=r.getElementsByTagName('state')[0].childNodes[0].data
            jobnum=r.getElementsByTagName('JB_job_number')[0].childNodes[0].data
            jobtime='not set'
            if(jobstate=='r'):
                jobtime=r.getElementsByTagName('JAT_start_time')[0].childNodes[0].data
            elif(jobstate=='dt'):
                jobtime=r.getElementsByTagName('JAT_start_time')[0].childNodes[0].data
            else:
                jobtime=r.getElementsByTagName('JB_submission_time')[0].childNodes[0].data

            print(jobnum, '\t', jobown.ljust(16), '\t', jobname.ljust(16),'\t', jobstate,'\t',jobtime)
        except Exception as e:
            print(e)

fakeqstat(runjobs)

For Python 2:

#!/usr/bin/python
import xml.dom.minidom
import os
import sys
import string
#import re


f=os.popen('qstat -u \* -xml -r')

dom=xml.dom.minidom.parse(f)


jobs=dom.getElementsByTagName('job_info')
run=jobs[0]

runjobs=run.getElementsByTagName('job_list')


def fakeqstat(joblist):
        for r in joblist:
                jobname=r.getElementsByTagName('JB_name')[0].childNodes[0].data
                jobown=r.getElementsByTagName('JB_owner')[0].childNodes[0].data
                jobstate=r.getElementsByTagName('state')[0].childNodes[0].data
                jobnum=r.getElementsByTagName('JB_job_number')[0].childNodes[0].data
                jobtime='not set'
                if(jobstate=='r'):
                        jobtime=r.getElementsByTagName('JAT_start_time')[0].childNodes[0].data
                elif(jobstate=='dt'):
                        jobtime=r.getElementsByTagName('JAT_start_time')[0].childNodes[0].data
                else:
                        jobtime=r.getElementsByTagName('JB_submission_time')[0].childNodes[0].data



                print  jobnum, '\t', jobown.ljust(16), '\t', jobname.ljust(16),'\t', jobstate,'\t',jobtime


fakeqstat(runjobs)