How to verify the usage of the GPU?

Question

How can I verify that CNTK is using the GPU? I have installed the CNTK-2-0-beta7-0-Windows-64bit-GPU-1bit-SGD binaries on my machine. But, when I try to run this from Python:

from cntk.device import set_default_device, gpu
set_default_device(gpu(0))

I get:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-8-eca77b3090be> in <module>()
      1 from cntk.device import set_default_device, gpu
----> 2 set_default_device(gpu(0))

C:\local\Anaconda3-4.1.1-Windows-x86_64\envs\cntk-py34\lib\site-packages\cntk\device.py in gpu(device_id)
     74         :class:`~cntk.device.DeviceDescriptor`: GPU device descriptor
     75     '''
---> 76     return cntk_py.DeviceDescriptor.gpu_device(device_id)
     77 
     78 def use_default_device():

ValueError: Specified GPU device id (0) is invalid.

Adding some more information today:

This is the result from running NVidia_smi.exe

C:\Program Files\NVIDIA Corporation\NVSMI>nvidia-smi.exe
Thu Jan 12 20:38:30 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 369.61                 Driver Version: 369.61                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GPU        WDDM  | 0000:01:00.0     Off |                  N/A |
| N/A   51C    P0     2W /  N/A |    864MiB /  1024MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

After restarting the kernel in a Jupyter Notebook, I get:

import cntk as C
if C.device.default().type() == 0:
    print('running on CPU')
else:
    print('running on GPU')

running on CPU

However today I was able to run:

from cntk.device import set_default_device, gpu
set_default_device(gpu(0))

import cntk as C
if C.device.default().type() == 0:
    print('running on CPU')
else:
    print('running on GPU')

running on GPU

Should the GPU be the default on a GPU machine, or do you need to explicitly set it?

Answer 1

This sounds like an intermittent failure. This can happen on some laptops such as the Surface Book which have two GPUs, one from NVIDIA and an integrated one, and the laptop has shutdown the NVIDIA GPU to conserve energy, e.g. when it is running on battery.

Regarding default behavior, by default CNTK will choose the best available device and if it is a GPU it will lock it so no other process can use it. If you explicitly use set_default_device(gpu(0)) then the GPU won't get locked and other processes can use it.