WCF Trace Log analysis - help

Question

I am having trouble deciphering a WCF trace file, and I hope someone can help me determine where in the pipeline I am incurring latency. The trace for "Processing Message XX" is shown below, where there appears to be 997ms delay between the Activity Boundary and the transfer to "Process Action" where my service code is executed (which takes approx 50ms).

First, I am unsure whether I am right in understanding the "Time" column to represent start time for the activity item. I believe this to be the case because, drilling into the "Processing action" trace displays a list of activities with the first timestamp equal to the timestamp shown in the above trace for the "Processing action" item.

My primary question is this: how do I determine what is happening during this 997ms time span? As I read about the service trace viewer, it seems that this activity type involves "transport or security processing", which leads me to believe it is a network issue, but I cannot be sure.

In case it is relevant, below is a snapshot of the drill-down to "Process action" trace.

Does anyone have some insight on how to drill further into this activity to pinpoint the cause of delay?

(I should mention that the response time varies from ~60ms to over a full second, and only seems to do so in a specific environment, which further leads me to the idea of a networking issue)

Thank you in advance!

Answer 1

I was having the identical problem. My transfer times ranged from 100's of milliseconds to 4 seconds. I installed Wireshark on the server and saw numerous network packet transmission errors. It was impressive that the network stack could sort it all out and the messages eventually went through. Eventually I noticed that the "Speed and Duplex" setting for the server NIC driver was set at 100Mb Full. The test client was at Auto and there were a couple of switches between them. I would think that all the devices could sort this out, but evidently not. Changing the server value to Auto resolved the network errors and the trace transfer delays went away.