Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to kill process on GPUs with PID in nvidia-smi using keyword?

How to kill running processes on GPUs for a specific program (e.g. python) in terminal? For example two processes are running with python in the top picture and kill them to see the bottom picture in nvidia-smi

For example two processes are running with python in the top picture and kill them to see the bottom picture in nvidia-smi

like image 882
salehinejad Avatar asked May 05 '18 20:05

salehinejad


People also ask

How do I kill a process on my GPU?

Find the process using your GPU resources, click on it, and press the End task button at the bottom right.

How do I stop apps from running on my GPU?

How Do I Stop An App From Using A Dedicated Gpu? Click Control + Shift + Escape on the keyboard to open Windows Task Manager. Right-click on a window that's empty on the taskbar and choose Task Manager from that window's options list. You can disable viewing GPU usage by default.


4 Answers

The accepted answer doesn't work for me, probably because nvidia-smi has different formats across different versions/hardware.

I'm using a much cleaner command:

nvidia-smi | grep 'python' | awk '{ print $3 }' | xargs -n1 kill -9

You can replace $3 in the awk expression to fit your nvidia-smi output. It is the n-th column in which the PIDs occur.

like image 177
Ainz Titor Avatar answered Oct 19 '22 01:10

Ainz Titor


You can grep python in the nvidia-smi and then pass the PID to the kill -9 command, e.g.

sudo kill -9 $( nvidia-smi | grep 'python' | sed -n 's/|\s*[0-9]\s([0-9])\s.*/\1/p' | sed '/^$/d')

like image 10
salehinejad Avatar answered Oct 19 '22 00:10

salehinejad


Use nvidia-smi or top command to see processes running and to kill command:

sudo kill -9 PID
like image 8
Snehal Rajput Avatar answered Oct 19 '22 01:10

Snehal Rajput


As one of other answers suggest you can use: (replace 5 with the column number where process id exists)

nvidia-smi | grep 'python' | awk '{ print $5 }' | xargs -n1 kill -9

If you might have to use this a lot you can create an alias for the command: to do that do this you should edit ~/.bash_aliases file:

nano ~/.bash_aliases

and add the following line to it and save the file:

alias killgpuprocess="nvidia-smi | grep 'python' | awk '{ print $5 }' | xargs -n1 kill -9"

then (just needed this time):

source ~/.bashrc

Then if you run

killgpuprocess

it will kill the existing processes on GPU(s).

like image 3
Sadra Naddaf Avatar answered Oct 19 '22 01:10

Sadra Naddaf