What's the difference between nvidia-smi Memory-Usage and GPU Memory Usage?

Question

Running the nvidia-smi command displays for example:

What's the difference between Memmory-Usage and GPU Memory Usage?

Answer 1

Memory-Usage indicates:

used memory/total memory

Used memory is how much of the GPU's memory is in use. Total Memory is how much is available, if nobody were using any of it.

GPU Memory Usage simply breaks this down by process.

This output apparently has one process running, and that process is using 10557 megabytes of GPU 0 memory. That is essentially the same as the 10570 number reported above it. Yes, there is a small difference, it is due to GPU memory consumption not associated with that process (e.g. in use by CUDA itself).