How to get a Windows symbol server set up

Question

I have a spare server on the network that I'd like to have host all of our build symbols. I already know how to set up a symbol cache on my local development system and pointing my debugger (e.g. Windbg) to that path in 'Symbol File Path'. I presumed that I could do the same with another system on the network that has its own symbol cache (e.g. \\host\symbols).

I set up the server and enabled file sharing on the directory, so I'm able to access it remotely through Windows Explorer. However, when I try pointing to it in WinDbg, it doesn't seem to pick up the symbols there.

The Symbol File Path is set up like this:

srv*\\192.168.1.20\symbolpath*http://msdl.microsoft.com/download/symbols

It seems like I'm not configuring it correctly on the server -- is there a step that I may be missing?

Answer 1

There are a several things to know when setting up a symbol server and/or symbol network share.

WinDbg symbol load order

Symbols are searched in the symbol path from the beginning to the end, i.e. in the symbol path C:\a;C:\b, it will first look in C:\a and then in C:\b. While this does not really matter, it influences performance a lot. If you have your own symbols, always put them first, so you're saving the HTTP round-trip to the Microsoft server.

Symbol store types

There are three symbol store types:

• Local store (directory on your disk)
• Server store (network share)
• Symbol server / HTTP store (with an HTTP URL)

Symbol store tiers

You can have three types of symbol stores and you should not mix them in a single directory:

• 0-tier or simply a flat list of PDB files, usually created as output of a build script or a copy/paste operation.
• 2-tier: Symbols are stored as <filename>.pdb\<hash>\<filename>.pdb. You recognize a 2-tier symbol store from an existing empty (0 byte) pingme.txt file and a 000Admin folder. Don't delete those.
• 3-tier: Symbols are stored as <fi>\<filename>\<hash>\<filename>.pdb>. You recognize a 3-tier symbol store from the empty (0 byte) index2.txt file. Don't delete it. The 3-tier store is supposed to increase performance.

You can put symbols from a 0-tier store to a 2- or 3-tier store using symstore.exe which comes with WinDbg. You can convert a 2-tier store into a 3-tier store using convertstore.exe. In case of errors, see Convertstore Errors.

Creating the "symbol server"

What you have set up is not a symbol server, it is a server symbol store, because you use (and want to use) a network share, not a HTTP web server. The following are the steps to set it up:

1. Create a new empty directory on the server
2. Share the folder with write access, if you want to add the symbols from a different machine. Read access should be sufficient if you add symbols from the server itself (e.g. if the server is a Continuous Integration server which builds the executables).
3. Run symstore add /3 /f "Filename.pdb" /s "\\server\symbols" /t "Title" if you want to add symbols from a different machine or use /s "C:\share\symbols" if you add them locally.

Repeat step 3 for all versions of PDB files you like to add. You can also use wildcards like *.pdb. Ideally you integrate that step into your build process.

Using the network share in WinDbg

For performance reasons, developers want to cache your own symbols from the network locally as well as the Microsoft symbols. Therefore, let's create such a local cache first:

.sympath cache*C:\Symbols

I typically let the cache folder be compressed by NTFS, because symbols compress quite well.

Next, let's find own symbols first to avoid the round-trip to Microsoft:

.sympath+ \\server\symbols

Finally, try to download everything else from Microsoft:

.symfix+

If you learn a bit about WinDbg Workspaces, you can save the symbol path setting in a workspace so you needn't type all this in each debugging session. Unfortunately it does not work if you put it all in one line separated by semicolon (I don't really understand why), but you can type .sympath now and copy the result. It should be

cache*c:\symbols;\\server\symbols;SRV*http://msdl.microsoft.com/download/symbols

Potential problems

I could not reproduce this now, but I remember some problems. The reason was: WinDbg will not ask for credentials when accessing the network share. The workaround is: if you don't receive symbols from \\server\symbols, open that network share in Windows Explorer. Explorer will ask for credentials and they will be cached by Windows and thus be used by WinDbg implicitly.