Determine total CPU count after qsub within PBS script

Question

For a PBS script called with qsub, I want to know how many total CPU's have actually been allocated in case the number defined in the PBS file is overwritten by inputs from the command line. For example with the following pbs script file:

jobscript.pbs:

#!/bin/bash
#PBS -N test_run
#PBS -l nodes=32
#PBS -l walltime=06:00:00
#PBS -j oe
#PBS -q normal
#PBS -o output.txt

cd $PBS_O_WORKDIR

module load gcc-openmpi-1.2.7
time mpiexec visct

This script could be run with just 16 CPU's (instead of 32) using the following command line:

$ qsub -l nodes=2:ppn=8 jobscript.pbs

So I would like a robust method for determining how many CPU's are actually available from within the script.

Answer 1

I was able to answer my own question with the following solution using the $PBS_NODEFILE environment variable which contains the path to a file listing information about the available nodes:

jobscript.pbs:

#!/bin/bash
#PBS -N test_run
#PBS -l nodes=32
#PBS -l walltime=06:00:00
#PBS -j oe
#PBS -q normal
#PBS -o output.txt

# This finds out the number of nodes we have
NP=$(wc -l $PBS_NODEFILE | awk '{print $1}')
echo "Total CPU count = $NP"

Thanks to "Source" after much online searching.

Answer 2

MasterHD I know you have found your answer but I thought I would share another way

This code is longer but it helps for my specific needs. I actually use pbsnodes commands. Below is a snippet of my code.

@nodes_whole =`pbsnodes -av -s $server | grep "pcpus" `;
$nodes_count = ` pbsnodes -av -s $server | grep "pcpus" | wc -l `;
while($i < $nodes_count){
    @cpu_present = split(/\s+/, $nodes_whole[$i]);
    $cpu_whole_count += $cpu_present[3];
    $i++;
}

I do this because in my script I check things like the amount of cpus , which varies depending on the node the cpus maybe be 4, 8, 16. Also I have multiple clusters which are always changing size and I don't want the script have specific cluster or node info hard coded. Mainly, I do this because when a user submits a job I check to see how many resources they can use . If say they want use a queue and request 200 cpus but on cluster A their job will be queued my script can tell them they will be queued but would not be on cluster b or d. So then they have the option to change before they submit.

I also use it to check for nodes down:

@nodes_down=`pbsnodes -l -s $server `;

I see what resources are in use:

@nodes_used=`pbsnodes -av -s $server | grep "resources_assigned.ncpus" `;

Also in one case I have two clusters running off one head node while I wait for hardware. In that case I check to see what cluster the node is assigned to and then do a count based on the node assigned to that cluster. That way all the users see is another cluster and use the script they way they would for any of the other clusters.

I just mention because I have found a lot of useful ways to use the pbsnodes and it worked well for my particular needs.