Correlate MSMQ End-to-End trace with WCF Trace and application level logging

Question

Background:

I'm troubleshooting a problem where messages sent by WCF over transactional MSMQ (with netMsmqBinding) seem to disappear. The code that uses WCF is in a third-party assembly which I cannot change. I have few clues to what the problem is, but plan to enable various tracing capabilities in order to pin-point where the problem relies.

Context:

1. I have enabled MSMQ End-to-End Tracing. It logs two events for every message that gets sent.

   • One event when a message is written to the outgoing queue. This message contain the MSMQ message id (which is composed by a guid and an integer, ie 7B476ADF-DEFD-49F2-AF5A-0CF27C5152C0\6481271).
   • Another event when that message is sent across the network.

2. I have enabled verbose WCF Tracing.

3. I also have application level logging that logs a message IDs defined by the application code (let's call this the "application message id").

4. I have enabled positive and negative source journaling on the MSMQ messages that get sent.

5. I have enabled journaling on the receiving queue.

Problem:

When messages go missing, I know the missing message's application id (it's logged by the sending side). I would now like to look at the End-to-End trace to see whether the message was written to the outgoing queue or not.

How can I correlate the events in the End-to-End trace with the application level logs and WCF traces?

Ideas:

• When sending a MSMQ message using the managed MSMQ API in System.Messaging, the message's MSMQ id is available after the message is sent. However, I have not found a way to log this when WCF is performing the send operation. The WCF trace logs a MSMQMessageId guid, but this value is, surprisingly, not the actual MSMQ id as I guessed it would be. Is it possible to access the actual MSMQ message id and log it?

• Log the native thread id in the application log along with the application level id and a time stamp. The native thread id is logged to the End-to-End trace by MSMQ, so this might actually be sufficient to correlate. This is plan B for me if I don't find a more elegant solution.

Answer 1

You sounds like you're on the right track. However you could bump up a bit with this:

Using SvcConfigEditor.exe

1. Configure WCF Verbose Tracing for Propagate ACtiveity and Activity tracing
2. Configure WCF MessageLogging for "Malformed Messages, Service Messages, Transport Messages"

Use LogEntireMessage

In End to End, trace it All

Make sure you enable these *.config on BOTH sides, yours and the 3rd party executable.

Collect your logs files, and add them ALL to SvcTraceViewer.exe