Redirect output to different files for sun grid engine array jobs (SGE)

Question

I am trying to execute a sun grid engine array job and I need a different output file per task: test.1.out test.2.out,etc..

If I write my script like this:

#!/bin/bash
#$ -S /bin/bash
#$ -N name
#$ -t 1-4000
#$ -o /home/myuser/out/test.$TASK_ID.out
#$ -e /home/myuser/err/test.$TASK_ID.err
#$ -cwd
#$ -V
<bash commands here>

The files are called $TASK_ID.out (1.out,2.out,etc..) and at /home/myuser/

And if I write my script like this:

#!/bin/bash
#$ -S /bin/bash
#$ -N name
#$ -t 1-4000
#$ -o /home/myuser/out/
#$ -e /home/myuser/err/
#$ -cwd
#$ -V
<bash commands here>

The files are located at the correct folder but evidently with the default name: name.o$JOB_ID.$TASK_ID

What should I do?

Answer 1

If having the specifically-named output files written in real-time is important, you might circumvent SGE's built-in output collection and invoke your bash commands like

$command 1>>/home/myuser/out/test.$SGE_TASK_ID.out 2>>/home/myuser/err/test.$SGE_TASK_ID.err

to capture stdout and stderr.

Both examples seems pretty close to what you want - any reason not to simply move/rename the output files after the fact?