Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Trace for function name from the output of cuda-memcheck

I'm running to cuda-memcheck to debug my code and the output is as follows

========= Program hit cudaErrorCudartUnloading (error 29) due to "driver shutting down" on CUDA API call to cudaFree. 
=========     Saved host backtrace up to driver entry point at error
=========     Host Frame:/usr/lib/x86_64-linux-gnu/libcuda.so.1 [0x2e40d3]
=========     Host Frame:./nmt [0x53526]
=========     Host Frame:./nmt [0xfbd9]
terminate called after throwing an instance of '=========     Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x3c259]
=========     Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x3c2a5]
=========     Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0xfc) [0x21ecc]
thrust::system::system_error'
=========     Host Frame:./nmt [0x530a]
=========
  what():  driver shutting down
========= Error: process didn't terminate successfully
========= Internal error (20)
========= No CUDA-MEMCHECK results found

Is it possible to tell from the line Host Frame:./nmt [0x53526] where is broken in the code? If so, how can I do that?

like image 890
Hieu Pham Avatar asked Nov 01 '22 01:11

Hieu Pham


1 Answers

As @talonmies indicated (I suspect he will not mind if I post a CW answer), the cuda-memcheck tool provides additional stack back tracing capability, which can be enabled with the --show-backtrace switch added to the command line.

The back trace may consist of both host and device functions (i.e. host and device back traces.)

If the application has been also compiled with host debug symbol information (e.g. -g on linux) then cuda-memcheck can show function names for the host functions in the host backtrace.

Additional usage information is available in the documentation.

like image 157
Robert Crovella Avatar answered Nov 10 '22 04:11

Robert Crovella