Use Bash variable within SLURM sbatch script

Question

I'm trying to obtain a value from another file and use this within a SLURM submission script. However, I get an error that the value is non-numerical, in other words, it is not being dereferenced.

Here is the script:

#!/bin/bash
# This reads out the number of procs based on the decomposeParDict                                                                                          
numProcs=`awk '/numberOfSubdomains/ {print $2}' ./meshModel/decomposeParDict`
echo "NumProcs = $numProcs"

#SBATCH --job-name=SnappyHexMesh                                                                                                                            
#SBATCH --output=./logs/SnappyHexMesh.log                                                                                                                   
#                                                                                                                                                           
#SBATCH --ntasks=`$numProcs`                                                                                                                                
#SBATCH --time=240:00                                                                                                                                       
#SBATCH --mem-per-cpu=4000                                                                                                                                  

#First run blockMesh                                                                                                                                        
blockMesh

#Now decompose the mesh                                                                                                                                     
decomposePar

#Now run snappy in parallel                                                                                                                                 
mpirun -np $numProcs snappyHexMesh -parallel -overwrite

When I run this as a normal Bash shell script, it prints out the number of procs correctly and makes the correct mpirun call. Thus the awk command parses out the number of procs correctly and the variable is dereferenced as expected.

However, when I submit this to SLURM using:

sbatch myScript.sh

I get the error:

sbatch: error: Invalid numeric value "`$numProcs`" for number of tasks.

Can anyone help with this?

Answer 1

This won't work. What happens when you run

sbatch myscript.sh

is that slurm parses the script for those special #SBATCH lines, generates a job record, stores the batch script somewhere. The batch script is executed only later when the job runs.

So you need to structure you workflow in a slightly different way, and first calculate the number of procs you need before submitting the job. Note that you can use something like

sbatch -n $numProcs myscript.sh

, you don't need to autogenerate the script (also, mpirun should be able to get the number of procs in your allocation automatically, no need to use "-np").

Answer 2

Slurm stops processing #SBATCH directives on the first line of executable code in a script. For users whose #SBATCH directives are not dependent on the code they're trying to run above those directives, just put the #SBATCH lines at the top.

See the other answer for a workaround/solution if, as with OP, your sbatch options are dependent on the commands you've placed above them.

The batch script may contain options preceded with "#SBATCH" before any executable commands in the script. sbatch will stop processing further #SBATCH directives once the first non-comment non-whitespace line has been reached in the script.

From the sbatch docs, my emphasis.