Nvidia visual studio Nsight CPU and GPU debugging

Question

The NVIDIA Nsight Visual Studio Edition does not seem to be capable of debugging CPU (host code) and GPU (cuda code) at the same time. With the Nsight Eclipse Edition (or cuda-gdb) this is quite simple, for example, you can "step in" to a CUDA kernel from the host execution. How to do the same with Visual Studio?

Answer 1

From the Nsight manual

It says

Use a separate Visual Studio instance to debug the host portion of a target application. If you wish to debug the host portion of your CUDA application while the CUDA Debugger is attached, you must attach using a different Visual Studio instance. Attaching the same instance of Visual Studio to debug both the host portion and the device portion of a target application will cause the debuggers to conflict. The result is that the target application and the CUDA Debugger hang while being blocked by operations of the native debugger.

So to debug a CUDA application here are the steps:

1. Open Visual Studio (VS instance #1) and set a breakpoint in a CUDA kernel and click "Start CUDA Debugging." This will start the application instance and stop where u set the breakpoint.
2. Open another instance of Visual Studio (VS instance #2) (needs to be run as admin for some reason) and attach to the process you started in step 1.
3. In VS instance #2 browse to the file you wish to do your CPU debugging and set a breakpoint.
4. In VS instance #1 continue the execution (should fall out of the current CUDA kernel). At this point the CPU breakpoint you set in VS instance #2 should be hit.

---

Extra credit: Debugging your CUDA application remotely. Steps:

1. On the target machine run the msvsmon.exe (remote debugger) as admin. This can be found at C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Remote Debugger\x64.
2. On the host machine (the one from which you want to control the debugging), open VS and make sure the Nsight debugger is pointed to the remote machine (Nsight User Properties > Launch > Connection Name). NOTE: the Nvidia Nsight monitor must be running on the target machine for this to work.
3. Do steps 1 and 2 from the previous (non-remote debugging) section. In step 2 you will need to point to the remote machine for CPU debugging (Debug>Attach to Process>Qualifier should be set to the target machine name or IP).
4. The non-remote debugging steps 3 and 4 apply apply here as well.

NOTE: It looks as if VS's remote CPU debugging is not as capable as its local CPU debugging. For example, when you mouse-over CPU variables, there values to not appear as you would expect when you are doing local debugging.

STILL UNANSWERED: Is it possible to start debugging the host code before hitting a CUDA breakpoint? It seems like a big limitation that you can only debug host code following your first CUDA kernel. What if you wish to debug the host code before the first CUDA kernel?