Connection reset by peer (502) when contacting WCF service in IIS 7.5

Question

We have a WCF service that has been working fine for months. Just this morning, calls to this service began failing with the proxy server reporting "connection reset by peer" (502). Oddly enough there are no IIS log entries for the dropped connections, although we are able to detect them with WireShark. So it seems as if the thread is dying, leaving no trace behind.

The problematic web method was retrieving 100 database records. We found that by restricting the results to 20 records it works as expected. I suspect there is a timeout coming into play, but with no exception or traceback it's very hard to know.

Has anyone experienced this? Any suggestions on how to troubleshoot it?

Answer 1

You can turn on WCF diagnostics on your service to get more details about exceptions.

To get you going quickly - in your web (or app) config:

1) Add System.Diagnostics section anywhere under configuration element. You can replace path with which ever path you want the files to be stored at.

<system.diagnostics>
    <sources>
      <source name="System.ServiceModel.MessageLogging" switchValue="Warning, ActivityTracing">
        <listeners>
          <add type="System.Diagnostics.DefaultTraceListener" name="Default">
            <filter type="" />
          </add>
          <add name="ServiceModelMessageLoggingListener">
            <filter type="" />
          </add>
        </listeners>
      </source>
      <source name="System.ServiceModel" switchValue="Warning, ActivityTracing" propagateActivity="true">
        <listeners>
          <add type="System.Diagnostics.DefaultTraceListener" name="Default">
            <filter type="" />
          </add>
          <add name="ServiceModelTraceListener">
            <filter type="" />
          </add>
        </listeners>
      </source>
    </sources>
    <sharedListeners>
      <add initializeData="C:\temp\services_messages.svclog" type="System.Diagnostics.XmlWriterTraceListener, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" name="ServiceModelMessageLoggingListener" traceOutputOptions="LogicalOperationStack, DateTime, Timestamp, ProcessId, ThreadId, Callstack">
        <filter type="" />
      </add>
      <add initializeData="C:\temp\services_tracelog.svclog" type="System.Diagnostics.XmlWriterTraceListener, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" name="ServiceModelTraceListener" traceOutputOptions="LogicalOperationStack, DateTime, Timestamp, ProcessId, ThreadId, Callstack">
        <filter type="" />
      </add>
    </sharedListeners>
  </system.diagnostics>

2) Under system.ServiceModel add following:

<diagnostics wmiProviderEnabled="false">
      <messageLogging logEntireMessage="true" logMalformedMessages="true" logMessagesAtServiceLevel="true" logMessagesAtTransportLevel="true" />
    </diagnostics>

3) Under C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin\ start SvcTraceViewer.exe. Load both message trace (services_messages.svclog) and service trace log (services_tracelog.svclog). You can either drag drop files in the tool or open one then add another

4) Look for red bold letters for a problem.

If you want to make your experience editing the WCF configuration more palatable you can use SvcConfigEditor.exe which is found under same folder as SvcTraceViewer.exe (#3). Just open the config file and you should see Diagnostics folder which will allow you to start/stop and configure diagnostics.

Answer 2

As for troubleshooting, a WCF trace can sometimes help figuring these issues out.