Manually stopping Spark Workers

Question

Is there a way to stop a Spark worker through the terminal? I'm aware of the scripts: start-all.sh, stop-all.sh, stop-workers.sh, etc. However, everytime I run start-all.sh there seems to be residual workers from a previous Spark cluster instance that are also spawned. I know this because the Worker Id contains the date and timestamp of when the worker was created.

So when I run start-all.sh today, I see the same 7 or so workers that were created at the beginning of April.

Is there a way to kill these earlier workers? Or perhaps a way to grep for their process names?

Answer 1

This has happened to me in the past and what I usually do is:

1) Find the process id:

ps aux | grep spark

2) And kill it:

sudo kill pid1