Preserving Polymorphic Types in a WCF Service using JSON

Question

I have a C# WCF service using a webHttpBinding endpoint that will receive and return data in JSON format. The data to send/receive needs to use a polymorphic type so that data of different types can be exchanged in the same "data packet". I have the following data model:

[DataContract]
public class DataPacket
{
    [DataMember]
    public List<DataEvent> DataEvents { get; set; }
}

[DataContract]
[KnownType(typeof(IntEvent))]
[KnownType(typeof(BoolEvent))]
public class DataEvent
{
    [DataMember]
    public ulong Id { get; set; }

    [DataMember]
    public DateTime Timestamp { get; set; }

    public override string ToString()
    {
        return string.Format("DataEvent: {0}, {1}", Id, Timestamp);
    }
}

[DataContract]
public class IntEvent : DataEvent
{
    [DataMember]
    public int Value { get; set; }

    public override string ToString()
    {
        return string.Format("IntEvent: {0}, {1}, {2}", Id, Timestamp, Value);
    }
}

[DataContract]
public class BoolEvent : DataEvent
{
    [DataMember]
    public bool Value { get; set; }

    public override string ToString()
    {
        return string.Format("BoolEvent: {0}, {1}, {2}", Id, Timestamp, Value);
    }
}

My service will send/receive the sub-type events (IntEvent, BoolEvent etc.) in a single data packet, as follows:

[ServiceContract]
public interface IDataService
{
    [OperationContract]
    [WebGet(UriTemplate = "GetExampleDataEvents")]
    DataPacket GetExampleDataEvents();

    [OperationContract]
    [WebInvoke(UriTemplate = "SubmitDataEvents", RequestFormat = WebMessageFormat.Json)]
    void SubmitDataEvents(DataPacket dataPacket);
}

public class DataService : IDataService
{
    public DataPacket GetExampleDataEvents()
    {
        return new DataPacket {
            DataEvents = new List<DataEvent>
            {
                new IntEvent  { Id = 12345, Timestamp = DateTime.Now, Value = 5 },
                new BoolEvent { Id = 45678, Timestamp = DateTime.Now, Value = true }
            }
        };
    }

    public void SubmitDataEvents(DataPacket dataPacket)
    {
        int i = dataPacket.DataEvents.Count; //dataPacket contains 2 events, but both are type DataEvent instead of IntEvent and BoolEvent
        IntEvent intEvent = dataPacket.DataEvents[0] as IntEvent;
        Console.WriteLine(intEvent.Value); //null pointer as intEvent is null since the cast failed
    }
}

When I submit my packet to the SubmitDataEvents method though, I get DataEvent types and trying to cast them back to their base types (just for testing purposes) results in an InvalidCastException. My packet is:

POST http://localhost:4965/DataService.svc/SubmitDataEvents HTTP/1.1
User-Agent: Fiddler
Host: localhost:4965
Content-Type: text/json
Content-Length: 340

{
    "DataEvents": [{
        "__type": "IntEvent:#WcfTest.Data",
        "Id": 12345,
        "Timestamp": "\/Date(1324905383689+0000)\/",
        "Value": 5
    }, {
        "__type": "BoolEvent:#WcfTest.Data",
        "Id": 45678,
        "Timestamp": "\/Date(1324905383689+0000)\/",
        "Value": true
    }]
}

Apologies for the long post, but is there anything I can do to preserve the base types of each object? I thought adding the type hint to the JSON and the KnownType attributes to DataEvent would allow me to preserve the types - but it doesn't seem to work.

Edit: If I send the request to SubmitDataEvents in XML format (with Content-Type: text/xml instead of text/json) then the List<DataEvent> DataEvents does contain the sub-types instead of the super-type. As soon as I set the request to text/json and send the above packet then I only get the super-type and I can't cast them to the sub-type. My XML request body is:

<ArrayOfDataEvent xmlns="http://schemas.datacontract.org/2004/07/WcfTest.Data">
  <DataEvent i:type="IntEvent" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
    <Id>12345</Id>
    <Timestamp>1999-05-31T11:20:00</Timestamp>
    <Value>5</Value>
  </DataEvent>
  <DataEvent i:type="BoolEvent" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
    <Id>56789</Id>
    <Timestamp>1999-05-31T11:20:00</Timestamp>
    <Value>true</Value>
  </DataEvent>
</ArrayOfDataEvent>

Edit 2: Updated service description after Pavel's comments below. This still doesn't work when sending the JSON packet in Fiddler2. I just get a List containing DataEvent instead of IntEvent and BoolEvent.

Edit 3: As Pavel suggested, here is the output from System.ServiceModel.OperationContext.Current.RequestContext.RequestMessage.ToString(). Looks OK to me.

<root type="object">
    <DataEvents type="array">
        <item type="object">
            <__type type="string">IntEvent:#WcfTest.Data</__type> 
            <Id type="number">12345</Id> 
            <Timestamp type="string">/Date(1324905383689+0000)/</Timestamp> 
            <Value type="number">5</Value> 
        </item>
        <item type="object">
            <__type type="string">BoolEvent:#WcfTest.Data</__type> 
            <Id type="number">45678</Id> 
            <Timestamp type="string">/Date(1324905383689+0000)/</Timestamp> 
            <Value type="boolean">true</Value> 
        </item>
    </DataEvents>
</root>

When tracing the deserialization of the packet, I get the following messages in the trace:

<TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Verbose">
    <TraceIdentifier>http://msdn.microsoft.com/en-GB/library/System.Runtime.Serialization.ElementIgnored.aspx</TraceIdentifier>
    <Description>An unrecognized element was encountered in the XML during deserialization which was ignored.</Description>
    <AppDomain>1c7ccc3b-4-129695001952729398</AppDomain>
    <ExtendedData xmlns="http://schemas.microsoft.com/2006/08/ServiceModel/StringTraceRecord">
        <Element>:__type</Element>
    </ExtendedData>
</TraceRecord>

This message is repeated 4 times (twice with __type as the element and twice with Value). Looks like the type hinting information is being ignored then the Value elements are ignored as the packet is deserialized to DataEvent instead of IntEvent/BoolEvent.

Answer 1

Whenever dealing with serialization, try to first serialize an object graph to see the serialized string format. Then use the format to produce correct serialized strings.

Your packet is incorrect. The correct one is:

POST http://localhost:47440/Service1.svc/SubmitDataEvents HTTP/1.1
User-Agent: Fiddler
Host: localhost:47440
Content-Length: 211
Content-Type: text/json

[
  {
    "__type":"IntEvent:#WcfTest.Data",
    "Id":12345,
    "Timestamp":"\/Date(1324757832735+0700)\/",
    "Value":5
  },
  {
    "__type":"BoolEvent:#WcfTest.Data",
    "Id":45678,
    "Timestamp":"\/Date(1324757832736+0700)\/",
    "Value":true
  }
]

Note the Content-Type header also.

I've tried it with your code and it works perfectly (well, I've removed the Console.WriteLine and tested in debugger). All the class hierarchy is fine, all objects can be cast to their types. It works.

UPDATE

The JSON you've posted works with the following code:

[DataContract]
public class SomeClass
{
  [DataMember]
  public List<DataEvent> dataEvents { get; set; }
}

...

[ServiceContract]
public interface IDataService
{
  ...

  [OperationContract]
  [WebInvoke(UriTemplate = "SubmitDataEvents")]
  void SubmitDataEvents(SomeClass parameter);
}

Note that another high-level node is added to the object tree.

And again, it works fine with inheritance.

If the problem still remains, please post the code that you use to invoke the service, as well as exception details you get.

UPDATE 2

How strange... It works on my machine.

I use .NET 4 and VS2010 with the latest updates on Win7 x64.

I take your service contract, implementation and data contracts. I host them in a web application under Cassini. I have the following web.config:

<configuration>
  <connectionStrings>
    <!-- excluded for brevity -->
  </connectionStrings>

  <system.web>
    <!-- excluded for brevity -->
  </system.web>

  <system.webServer>
    <modules runAllManagedModulesForAllRequests="true"/>
  </system.webServer>
  <system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name="">
          <serviceMetadata httpGetEnabled="true" />
          <serviceDebug includeExceptionDetailInFaults="false" />
        </behavior>
      </serviceBehaviors>
      <endpointBehaviors>
        <behavior name="WebBehavior">
          <webHttp />
        </behavior>
      </endpointBehaviors>
    </behaviors>
    <serviceHostingEnvironment multipleSiteBindingsEnabled="true" />
    <services>
      <service name="WebApplication1.DataService">
        <endpoint address="ws" binding="wsHttpBinding" contract="WebApplication1.IDataService"/>
        <endpoint address="" behaviorConfiguration="WebBehavior"
           binding="webHttpBinding"
           contract="WebApplication1.IDataService">
        </endpoint>
        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/>
      </service>
    </services>
  </system.serviceModel>
</configuration>

Now I make the following POST by Fiddler2 (important: I've renamed the namespace of the derived types to match my case):

POST http://localhost:47440/Service1.svc/SubmitDataEvents HTTP/1.1
User-Agent: Fiddler
Content-Type: text/json
Host: localhost:47440
Content-Length: 336

{
    "DataEvents": [{
        "__type": "IntEvent:#WebApplication1",
        "Id": 12345,
        "Timestamp": "\/Date(1324905383689+0000)\/",
        "Value": 5
    }, {
        "__type": "BoolEvent:#WebApplication1",
        "Id": 45678,
        "Timestamp": "\/Date(1324905383689+0000)\/",
        "Value": true
    }]
}

Then I have the following code in the service implementation:

public void SubmitDataEvents(DataPacket parameter)
{
  foreach (DataEvent dataEvent in parameter.DataEvents)
  {
    var message = dataEvent.ToString();
    Debug.WriteLine(message);
  }
}

Note that debugger shows the items details as DataEvents, but string representations and the first item in the details clearly show that all sub-types have been deserialized well:

And debug output contains the following after I hit the method:

IntEvent: 12345, 26.12.2011 20:16:23, 5
BoolEvent: 45678, 26.12.2011 20:16:23, True

I've also tried running it under the IIS (on Win7) and everything works fine too.

I've had only the base type deserialized after I corrupted the packet by deleting one underscore from the __type field name. If I modify the value of __type, the call will crash during deserialization, it won't hit the service.

Here is what you could try:

1. Make sure you don't have any debug messages, exceptions, etc (check Debug Output).
2. Create a new clean web application solution, paste the required code and test if it works there. If it does, then your original project must have some weird configuration settings.
3. In debugger, analyze System.ServiceModel.OperationContext.Current.RequestContext.RequestMessage.ToString() in the Watch window. It will contain the XML message translated from your JSON. Check if it is correct.
4. Check if you have any pending updates for .NET.
5. Try tracing WCF. Although it doesn't seem to emit any warnings for messages with wrong __type field name, it may happen that it will show you some hints for your issues reasons.

My RequestMessage

Seems like here is the track of the issue: while you have __type as element, I have it as attribute. Supposedly, your WCF assemblies have a bug in JSON to XML translation

<root type="object">
  <DataEvents type="array">
    <item type="object" __type="IntEvent:#WebApplication1">
      <Id type="number">12345</Id>
      <Timestamp type="string">/Date(1324905383689+0000)/</Timestamp>
      <Value type="number">5</Value>
    </item>
    <item type="object" __type="BoolEvent:#WebApplication1">
      <Id type="number">45678</Id>
      <Timestamp type="string">/Date(1324905383689+0000)/</Timestamp>
      <Value type="boolean">true</Value>
    </item>
  </DataEvents>
</root>

I've found the place where __type is processed. Here it is:

// from System.Runtime.Serialization.Json.XmlJsonReader, System.Runtime.Serialization, Version=4.0.0.0
void ReadServerTypeAttribute(bool consumedObjectChar)
{
  int offset;
  int offsetMax; 
  int correction = consumedObjectChar ? -1 : 0;
  byte[] buffer = BufferReader.GetBuffer(9 + correction, out offset, out offsetMax); 
  if (offset + 9 + correction <= offsetMax) 
  {
    if (buffer[offset + correction + 1] == (byte) '\"' && 
        buffer[offset + correction + 2] == (byte) '_' &&
        buffer[offset + correction + 3] == (byte) '_' &&
        buffer[offset + correction + 4] == (byte) 't' &&
        buffer[offset + correction + 5] == (byte) 'y' && 
        buffer[offset + correction + 6] == (byte) 'p' &&
        buffer[offset + correction + 7] == (byte) 'e' && 
        buffer[offset + correction + 8] == (byte) '\"') 
    {
      // It's attribute!
      XmlAttributeNode attribute = AddAttribute(); 
      // the rest is omitted for brevity
    } 
  } 
}

I've tried to find the place where the attribute is used to determine the deserialized type, but to no luck.

Hope this helps.

Answer 2

Thanks to Pavel Gatilov, I've now found the solution to this problem. I'll add it as separate answer here for anyone who may be caught out by this in future.

The problem is that the JSON deserializer doesn't seem to be very accepting of whitespace. The data in the packet that I was sending was "pretty printed" with line breaks and spaces to make it more readable. However, when this packet was deserialized, this meant that when looking for the "__type" hint, the JSON deserializer was looking at the wrong part of the packet. This meant that the type hint was missed and the packet was deserialized as the wrong type.

The following packet works correctly:

POST http://localhost:6463/DataService.svc/SubmitDataEvents HTTP/1.1
User-Agent: Fiddler
Content-Type: text/json
Host: localhost:6463
Content-Length: 233

{"DataEvents":[{"__type":"IntEvent:#WebApplication1","Id":12345,"Timestamp":"\/Date(1324905383689+0000)\/","IntValue":5},{"__type":"BoolEvent:#WebApplication1","Id":45678,"Timestamp":"\/Date(1324905383689+0000)\/","BoolValue":true}]}

However, this packet doesn't work:

POST http://localhost:6463/DataService.svc/SubmitDataEvents HTTP/1.1
User-Agent: Fiddler
Content-Type: text/json
Host: localhost:6463
Content-Length: 343

{
    "DataEvents": [{
        "__type": "IntEvent:#WebApplication1",
        "Id": 12345,
        "Timestamp": "\/Date(1324905383689+0000)\/",
        "IntValue": 5
    }, {
        "__type": "BoolEvent:#WebApplication1",
        "Id": 45678,
        "Timestamp": "\/Date(1324905383689+0000)\/",
        "BoolValue": true
    }]
}

These packets are exactly the same apart from the line breaks and spaces.