cuda-memcheck, how to get from address to source code?

Question

I successfully used cuda-memcheck to get errors about wrong memory accesses. Compiling the cuda code with -g -G gave nice source locations like this:

========= Error: process didn't terminate successfully
========= Invalid __global__ read of size 1
=========     at 0x00000710 in /some/path/somefile.cu:117:some_function
=========     by thread (0,14,0) in block (1,16,0)
=========     Address 0x00abac20 is out of bounds

Now I tried to use the -l switch to get also memory leak information. Here, however, I get only addresses:

========= CUDA-MEMCHECK
========= Leaked 3630 bytes at 0x007d2800
=========
========= Leaked 14740 bytes at 0x008e0700
...
=========
========= LEAK SUMMARY: 11122140 bytes leaked in 39 allocations
========= ERROR SUMMARY: 0 errors
400 bytes at 0x005d2000

How can I get the actual code locations from this?

Answer 1

The addresses provided for the leak are not code addresses but rather data locations. Unfortunately it's not easy to see where these locations were allocated.

Given that the memory could have been allocated anywhere (remember that pointers can be passed around, aliased, etc.) the only way to check for leaks (i.e. allocated memory that is not freed) is when the program exits. So when your program exits cuda-memcheck checks for chunks of memory that were allocated but not freed and gives you the address of the chunk of memory, but it has no way to tie that back to when it was allocated.

Instead the easiest is to manually inspect your code to check that all cudaMalloc() calls have a matching cudaFree() call. This can, however, be quite a laborious process...