Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pass command line arguments via sbatch

Suppose that I have the following simple bash script which I want to submit to a batch server through SLURM:

#!/bin/bash  #SBATCH -o "outFile"$1".txt" #SBATCH -e "errFile"$1".txt"  hostname  exit 0 

In this script, I simply want to write the output of hostname on a textfile whose full name I control via the command-line, like so:

login-2:jobs$ sbatch -D `pwd` exampleJob.sh 1 Submitted batch job 203775 

Unfortunately, it seems that my last command-line argument (1) is not parsed through sbatch, since the files created do not have the suffix I'm looking for and the string "$1" is interpreted literally:

login-2:jobs$ ls errFile$1.txt  exampleJob.sh outFile$1.txt 

I've looked around places in SO and elsewhere, but I haven't had any luck. Essentially what I'm looking for is the equivalent of the -v switch of the qsub utility in Torque-enabled clusters.

Edit: As mentioned in the underlying comment thread, I solved my problem the hard way: instead of having one single script that would be submitted several times to the batch server, each with different command line arguments, I created a "master script" that simply echoed and redirected the same content onto different scripts, the content of each being changed by the command line parameter passed. Then I submitted all of those to my batch server through sbatch. However, this does not answer the original question, so I hesitate to add it as an answer to my question or mark this question solved.

like image 673
Jason Avatar asked Dec 30 '14 16:12

Jason


People also ask

How do you pass command line arguments?

To pass command line arguments, we typically define main() with two arguments : first argument is the number of command line arguments and second is list of command-line arguments. The value of argc should be non negative. argv(ARGument Vector) is array of character pointers listing all the arguments.

How do I pass a command line argument to a batch file?

In the batch script, you can get the value of any argument using a % followed by its numerical position on the command line. The first item passed is always %1 the second item is always %2 and so on. If you require all arguments, then you can simply use %* in a batch script.

How do I send command line arguments in Visual Studio?

To set command-line arguments in Visual Studio, right click on the project name, then go to Properties. In the Properties Pane, go to "Debugging", and in this pane is a line for "Command-line arguments." Add the values you would like to use on this line. They will be passed to the program via the argv array.


2 Answers

I thought I'd offer some insight because I was also looking for the replacement to the -v option in qsub, which for sbatch can be accomplished using the --export option. I found a nice site here that shows a list of conversions from Torque to Slurm, and it made the transition much smoother.

You can specify the environment variable ahead of time in your bash script:

$ var_name='1' $ sbatch -D `pwd` exampleJob.sh --export=var_name 

Or define it directly within the sbatch command just like qsub allowed:

$ sbatch -D `pwd` exampleJob.sh --export=var_name='1' 

Whether this works in the # preprocessors of exampleJob.sh is also another question, but I assume that it should give the same functionality found in Torque.

like image 70
MasterHD Avatar answered Sep 29 '22 04:09

MasterHD


Using a wrapper is more convenient. I found this solution from this thread.

Basically the problem is that the SBATCH directives are seen as comments by the shell and therefore you can't use the passed arguments in them. Instead you can use a here document to feed in your bash script after the arguments are set accordingly.

In case of your question you can substitute the shell script file with this:

#!/bin/bash sbatch <<EOT #!/bin/bash  #SBATCH -o "outFile"$1".txt" #SBATCH -e "errFile"$1".txt"  hostname  exit 0 EOT 

And you run the shell script like this:

bash [script_name].sh [suffix] 

And the outputs will be saved to outFile[suffix].txt and errFile[suffix].txt

like image 33
PouyaB Avatar answered Sep 29 '22 05:09

PouyaB