Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I terminate GNU parallel without killing running jobs?

I'm running a bunch of shell scripts like parallel -a my_scripts bash and at some point I decided I've run enough of them and would like to stop spawning new jobs, and simply let all the existing jobs finish. Put another way, I want to kill the parent process without killing the children.

There seems to be ways of controlling termination when first launching GNU parallel (for example if I know in advance I only want to run x jobs, then I can use --halt now,success=x argument), but I couldn't find how to control GNU parallel when it is already running.

Sure I can just CTRL+C to kill parallel, and rerun the jobs that were aborted, but I thought there might be a smarter way.

like image 523
Yibo Yang Avatar asked Jul 17 '17 15:07

Yibo Yang


1 Answers

Update:

If you have a new version of parallel >= 20190322: as per @Bowi's comment, "since 2019, it is no longer SIGTERM, it's SIGHUP instead:

kill -HUP $PARALLEL_PID

SIGTERM now terminates the children without letting them finish."

See:

  • https://superuser.com/questions/1660301/gnu-parallel-is-behaving-differently-on-sigterm-in-two-cygwin-installation
  • http://git.savannah.gnu.org/cgit/parallel.git/tree/NEWS#n687

Original answer:

I figured it out. The answer is to simply send SIGTERM to the parent parallel process (just killing its PID will do). parallel then responds with the following (in this case I have 4 jobs running):

parallel: SIGTERM received. No new jobs will be started.
parallel: Waiting for these 4 jobs to finish. Send SIGTERM again to stop now.

I dug it out of the man page:

COMPLETE RUNNING JOBS BUT DO NOT START NEW JOBS
If you regret starting a lot of jobs you can simply break GNU parallel, but if you want to make sure you do not have half-completed jobs you should send the signal SIGTERM to GNU parallel:

killall -TERM parallel

This will tell GNU parallel to not start any new jobs, but wait until the currently running jobs are finished before exiting.

like image 92
Yibo Yang Avatar answered Oct 25 '22 23:10

Yibo Yang