Is there a "one-liner" for submitting many jobs to SLURM (similar to LSF)?

Question

Can I submit "one-liners" to SLURM?

Using bsub from LSF and the standard Linux utility xargs, I can easily submit a separate job for uncompressing all of the files in a directory:

ls *.gz | sed 's/.gz$//g' | xargs -I {} bsub 'gunzip -c {}.gz > {}'

Using SLURM, I thought that srun or sbatch would be work, but to no avail:

ls *.gz | sed 's/.gz$//g' | xargs -I {}  srun 'gunzip -c {}.gz > {}'
gzip: srun: error: compute-node-01: task 0: Exited with exit code 1
stdin: unexpected end of file

ls *.gz | sed 's/.gz$//g' | xargs -I {}  sbatch 'gunzip -c {}.gz > {}'
sbatch: error: Unable to open file gunzip -c naive_S1_L001_R1_001.fastq.gz > naive_S1_L001_R1_001.fastq

I have seen bsub from LSF listed as equivalent to sbatch from SLURM, but so far it seems that they are only equivalent for submitting script files:

                  SLURM                    LSF
                  --------------------     ------------------
Job Submission    sbatch [script_file]     bsub [script_file]

Is there any other way to submit "one-liner" jobs with SLURM?

Answer 1

Try using the wrap option of sbatch. Something like the following:

ls *.gz | sed 's/.gz$//g' | xargs -I {}  sbatch --wrap="gunzip -c {}.gz > {}"

From the man page for `sbatch`:

--wrap=<command string>
       Sbatch will wrap the specified command string in  a  simple  "sh"  shell
       script,  and submit that script to the slurm controller.  When --wrap is
       used, a script name and arguments may not be specified  on  the  command
       line; instead the sbatch-generated wrapper script is used.